Re: [Boost-users] Subject: Formal Review of Proposed Boost.Process library starts tomorrow

Subject: Re: [Boost-users] Subject: Formal Review of Proposed Boost.Process library starts tomorrow
From: Jeff Flinn (TriumphSprint2000_at_[hidden])
Date: 2011-02-12 11:14:38

Boris Schaeling wrote:
> On Fri, 11 Feb 2011 15:01:35 +0100, Jeff Flinn 
> <TriumphSprint2000_at_[hidden]> wrote:
> 
>> [...]The myriad of stream behaviour, stream ends, and methods of 
>> specification to me do not best represent the problem domain.
>> [...]So having stdin, stdout, stderr Initializers with constructors or 
>> factory functions:
>>
>> stdin_from()
>> stdin_from(const path&)
>> stdin_from(const file_descriptor_source&)
>> stdin_from(const file_descriptor_ray&)
>>
>> stdout_to()
>> stdout_to(const path&)
>> stdout_to(const file_descriptor_sink&)
>> stdout_to(const file_descriptor_ray&)
>>
>> stdout_to()
>> stderr_to(const path&)
>> stderr_to(const file_descriptor_sink&)
>> stderr_to(const file_descriptor_ray&)
>>
>> is to me conceptually much simpler than dealing with stream behaviour, 
>> stream end and context classes. It also minimizes copying and heap 
>> allocated handles.
> 
> You basically replace boost::process::handle with various types? This 
> makes redirecting to a file very simple indeed.
> 
> Some stream behaviors would be still required though - somehow you need 
> to create a pipe for example? Stream behaviors would be called 
> differently but they would still return two stream ends. These could now 
> be objects of different types (so the return type is not necessarily 
> called stream_ends either).
> 
> What I like about your idea is that it's not required anymore to use two 
> objects (stream ends as they are currently called) if you really need 
> only one (for example to redirect to a file). I also like that a stream 
> behavior (or whatever it is called then) which creates for a example a 
> pipe wouldn't need to return a parent and child end but can return a 
> read and write end.

The Ray concept and it's model file_descriptor_ray as described in the 
paragraph that was clipped in your reply describes this.

"Ray, which is a connected pair of boost iostreams' Source and Sink 
concepts. Pipe is too general in that it conveys flow can be in either 
direction. So assuming unix still: "uses just one internal buffer for 
temporary storage of the flowing data, hence what goes in one direction 
overwrites whatever is a naive programmer tries to send backward" as 
stated in: http://goo.gl/RXY7p, makes Ray a more accurate term than 
pipe. Then a model of a Ray would be file_descriptor_ray, which 
comprises a boost iostream file_descriptor_sink and a 
file_descriptor_source."

'stderr_to(const file_descriptor_ray&)' and it's siblings provide 
connection to a Ray(which is defined to be a directed pipe). If the 
parent is chaining children(hope this doesn't get flagged by law 
enforcement), it doesn't need to keep the ray around once it's done 
launching the children. The parent then should be able to extract the 
unconnected source/sink of a ray from which to read/write from/to.

I'll try to post a file_descriptor_ray and corresponding initializer 
implementation later today.

> How would the library support inheriting file descriptors > 2 on POSIX? 
> With something like fd(int, const file_descriptor_source&) and fd(int, 
> const file_descriptor_sink&)?

I've limited my focus to dealing with std io to begin with, and haven't 
given it the thought it requires yet. If required, I'm sure an 
initializer could be provided to do what's necessary. I don't remember 
the previous discussions on just how important it is to go beyond std 
io. I'll try to dig up a link, but I remember reading that pipes in 
general are the last choice for interprocess communication on some OS's 
at least.

>> [...]Sorry, my thoughts didn't make it all the way out of my finger 
>> tips. I meant boost::process::child has a std::map<stream_id, handle>  
>> data member. Once the child process has been initialized/launched 
>> there is no need to know about any handles. Only the process id/handle 
>> is required when one needs to wait on the child process to complete. 
>> My view is that any stream definition info needing to be accessed 
>> after initialization does not belong in the child process class.
> 
> The stream definitions are in the context class. The map in the child 
> class is for stream ends to communicate with the child process. If you 
> create for example a pipe and want the child process to write to stdout, 
> the read end of the pipe is stored in this map.

I understand, and my view is that io entities should not be present in 
any form in a child process entity. The client code owns only those io 
entities it created to communicate with it's launched child processes, 
other io entities used to connect sibling children need only be 
temporaries. In fact often when launching gui apps as children, there is 
no stream io involved, inter process communication is done with 
boost::interprocess, asio, com,....

Thanks, Jeff