Re: [boost] [interprocess][process] Synchronizing libraries and efforts (+ bugs)

From: Ion Gaztañaga (igaztanaga_at_[hidden])
Date: 2008-08-31 17:20:10

Boris wrote:
> On Sun, 31 Aug 2008 18:23:30 +0200, Ion Gaztañaga <igaztanaga_at_[hidden]> 
> wrote:
> 
> Hi Ion,
> 
>> I recently needed s library to launch processes and I took a look to 
>> the recently announced Process 0.2 
>> (http://lists.boost.org/Archives/boost/2008/07/139322.php).
> 
> I'm currently working on Boost.Process 0.3. After having released 
> version 0.2 I had learnt that there was another version of Boost.Process 
> somewhere in Subversion which was more recent than version 0.1 which I 
> had used to create 0.2. With Boost.Process 0.3 all changes in all 
> snapshots will be merged again.

Nice to hear it.

>> [...]A process-launching library is a very nice utility for 
>> Interprocess and
>> I was thinking of adding this possibility for a future version (Boost 
>> 1.37 or 1.38) of the library. If Boost.Process is going forward, it 
>> would be glad to collaborate and develop a library that can satisfy 
>> all our needs and plays nicely with Interprocess.
> 
> Sure! I had picked up Boost.Process myself which was created in a Google 
> Summer of Project in 2006 by someone else.

Great!

>> [...]2) Ability to redirect child process stdin/stdout/stderr to 
>> /dev/null, to share the stdin/stdout of the parent or to redirect it 
>> to a file. And in the latter case, append or overwrite options 
>> (Boost.Process implements some of this options). I slightly modified 
>> Boost.Process to implement a rought version of file redirection.
> 
> This is already supported in Boost.Process 0.3.

Ok, glad to hear it.

>> [...]5) I don't know if streams are the right way to communicate with 
>> processes, because they don't offer non-blocking support.
> 
> I had added a method to access a file handle in version 0.2 I think. 
> That way it's possible to use asynchronous I/O (you need Boost.Asio 1.1 
> at least; I think this version supports I/O objects for file descriptors 
> the first time). There is no example yet but asynchronous I/O works as 
> far as I can tell.

Ok. I think non-blocking support should be built in the Process library. 
I understand async I/O should be tied with ASIO. I think Boost.Iostreams 
has some kind of non-blocking IO, but I'm not sure. Anyway, we would 
just need to add to systembuf new functions implementing non-blocking 
IO. Something basic like write/read functions. Then we can build 
something better above this with Iostreams: I wouldn't like 
Boost.Process to play nice with Iostreams, but also the option of not 
using Boost.Iostreams at all.

Another issue, maybe we should create different pipes depending on the 
request (at least in Windows):

-> Async I/O: named pipes
-> Blocking & non-blocking I/O: anonymous pipes (they need fewer 
resources according to MSDN).

>> 6) I think that process-launching interface should be similar to the 
>> soon to be standardized C++ futures.
> 
> I'm not sure to what you are referring (do you have a link?). However 
> the Launcher concept is replaced by a Context concept in 0.3. This was a 
> change in Subversion by whoever was doing the changes there (with 0.3 
> I'm really trying to get one version of Boost.Process again without 
> making any other changes).

See N2627:

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2627.html

You have a prototype written by Anthony Williams in 
Boost.Vault/Concurrent Programming. Obviously, I don't think we need to 
return an arbitrary value for processes. My first impression is that 
"child" (or whatever is called now ;-) ) should be similar to 
unique_future<int> in some basic operations so that any programmer that 
knows futures can start launching processes pretty soon. I don't think 
we can apply the rest of thread classes to processes (I don't think we 
can t propagate exceptions and returning arbitrary values would require 
some serious work with shared memory, that I prefer to avoid).

>> So I'm interested in a process-launching library and I offer my 
>> collaboration in case Boost.Process development needs help.
> 
> Can you wait two weeks? :) I should be done with version 0.3 until mid 
> September. I'm nearly done going through the source code and can send 
> you the .hpp files if you want? I still need to update the examples and 
> documentation though.

I can wait ;-) Since you are working hard on the library, I think it's 
better to wait until you consider the library mature enough. Feel free 
to contact me if you have any question.

>> PS: Some possible bugs found on Boost.Process 0.2 version (Windows):
> 
> Thanks, I'll go through the list later. There might be bugs in the POSIX 
> code, too, because I never tested the code. But as I said priority #1 is 
> currently merging all the snapshots again. :)

Yes, that's the most important task right now and thanks for pushing the 
library. I think it will cover a basic hole in the C++ community.

> Boris

Regards,

Ion