Re: [boost] [boost.process] 0.6 Alpha

Subject: Re: [boost] [boost.process] 0.6 Alpha
From: Damien Buhl (damien.buhl_at_[hidden])
Date: 2016-06-17 09:31:39

On 17/06/2016 11:45, Klemens Morgenstern wrote:
> [snip]
>>> So I guess, I wouldn´t use vfork as default, but it might be possible
>>> to add a property, which will cause the library to use that. I.e. you
>>> write:
>>>
>>> boost::process::child c("java.exe", "overhead.jar",
>>> boost::process::posix::use_vfork);
>>>
>>> But I'd need to be able to check if vfork is available, so I can
>>> disable the property if not.
>>>
>>> Would that be sufficient for your problem?
>> Naturally if the code still compiles on windows even though I'm using
>> boost::process::posix::use_vfork(_if_possible) then yes for me all will
>> be fine.
>> But on the other hand from a library design point of view, shouldn't the
>> library have the best smart default in terms of performance and overhead
>> on a given platform ? Instead of having a flag telling : please do it
>> the same but efficiently ? Because on linux vfork is nothing but
>> obsoleted and for a scenario of using execve looks better to me.
> Again: obsolete and now removed in the posix-standard and that's what
> I ought to go with. Please note, that I'm trying to provide two
> platforms: Posix & Windows. Not Linux & Windows. Thereby I want to
> provide the most common way for both platforms; and though I really
> appreciate your scenario, I would not consider it the common way.
>
> I think you underestimate the dangers of vfork; because you know, two
> processes sharing memory (including the stack!) can be rather fun.
> There's a reason it was removed and so it will be optional in
> boost.process.
>
> It would be a platform extension, so no, this would NOT compile with
> windows.  I thought about turning the platform extensions into NOPs ,
> but that's just too weird - especially since things like
> signal(SIGCHLD, ...) are also provided. There you should rather use
> the preprocessor and put an #ifdef there - then it's obvious what
> you're doing. Also I'd need a #define to know whether vfork is
> available, that I would probably provide. So you could then write
> something like that:
>
> child c("ls"
> #if defined(BOOST_POSIX_HAS_VFORK)
>       , posix::use_vfork
> #endif
>     );
>
> I guess I can check that via 'CLONE_VFORK'.
Yes that's fine, I just find it better when a library user don't have to
use #ifdef at all, that's why I was asking. But that's all the
difficulty of Boost.Process : it's pretty hard to abstract something
which is really different in the little detail.

Just for reference, this is what posix_spawn does in the glibc, which is
roughly equivalent to what Boost.Process provides via the child class,
they are also feared about using vfork, so you are surely right to avoid
it as much as possible :

In glibc /sysdeps/posix/spawni.c
>   /* Generate the new process.  */
>   if ((flags & POSIX_SPAWN_USEVFORK) != 0
>       /* If no major work is done, allow using vfork.  Note that we
>      might perform the path searching.  But this would be done by
>      a call to execvp(), too, and such a call must be OK according
>      to POSIX.  */
>       || ((flags & (POSIX_SPAWN_SETSIGMASK | POSIX_SPAWN_SETSIGDEF
>             | POSIX_SPAWN_SETSCHEDPARAM | POSIX_SPAWN_SETSCHEDULER
>             | POSIX_SPAWN_SETPGROUP | POSIX_SPAWN_RESETIDS)) == 0
>       && file_actions == NULL))
>     new_pid = __vfork ();
>   else
>     new_pid = __fork ();
>

The rationale of posix_spawn here is a good addition to what we discuss
I think :
http://pubs.opengroup.org/onlinepubs/009695399/functions/posix_spawn.html#tag_03_427_08

And as your goal is Posix + Windows, I understand you don't want to deal
with specialities of linux / solaris / ... so I think you're right it's
safer the default to be fork.

>> [snip]
>>
>> It's with an MMU but the parent process on this product is a
>> memory-hungry-monster-jvm and the fork fails.
> It seems a bit strange to me, that you would use boost.process here,
> instead of - you know - java.io.process. But I guess you want
> performance and your java-developers want a job, so JNI is the way to go?
>
It's off-topic but to explain: The fact is that they just reuse a C++
library we had for long, where we added a JNI interface for it, which is
launching at one point processes that are not developed internally and
that we cannot implement as library because we don't have the source code.

Additionally I believe that in the version of jvm implementation used
(oracle - arm - early 1.7) I think java.io.process does fork+exec, which
has been changed after because of similar issues:
http://bugs.java.com/view_bug.do?bug_id=6868160

Thanks for dealing with my wishes to vfork :)