Re: [boost] [boost.process] 0.6 Redesign

Subject: Re: [boost] [boost.process] 0.6 Redesign
From: Klemens Morgenstern (klemens.morgenstern_at_[hidden])
Date: 2016-04-20 12:32:46

Am 20.04.2016 um 17:50 schrieb Raphaël Londeix:
> Hi,
>
> Of course you can implement the example above very easily with the process
>> library:
>>
>
> Thanks, that's almost what I need, but it is not practical for stream
> initializations:
>
>      enum stream { std_in, std_out, std_err, dev_null };
>
>      struct options
>      {
>          // stderr redirection
>          boost::variant<
>              boost::filesystem::path,
>              boost::process::pipe,
>              stream> err;
>
>          ...
>      };
>
> How can I spawn a process, so that it will redirect stderr alternatively to
> a path, to a pipe, to stdout, or to /dev/null ? If I'm right, I will need
> to write explicitly 4 different calls to process::execute(). And it gets
> worse if we want to have the same flexibility for the stdin and stdout ...

I don't think so. You could built a class as follows (I think, didn't 
test this) - looks not as nice as it could be, but should do the job.

using stream = variant<
                decltype(process::null),
                decltype(process::close),
                filesystem::path,
                process::pipe,
                stream>; //< the last one would not work...

template<typename T> stream_vis : boost::static_visitor<child>
{
     std::string cmd;
     stream_vis(const std::string & cmd) cmd(cmd) {}
     template<typename T, typename U, typename V>
     child operator()(const T & in, const U & out, const V & err) const
     {
         return execute(cmd, std_in < in, std_out > out, std_err > err);
     }
};

child my_execute(const std::string &cmd,
                 const stream &in, const stream &out, const stream &err)
{
     stream_vis sv(cmd);
     return apply_visitor(sv, in, out, err);
}

I think this would be the way to go, though there could be some helper 
class for that.

>
> All in all, I'm not suggesting that you add support for boost::variant, but
> instead suggesting that boost.process could have one low-level generic way
> to spawn a process. As mentioned earlier, the Python subprocess library is
> great, its subprocess.Popen[1] constructor provides a way to do so.
>

I need the initializers to be different classes for the I/O. Or at least 
I'd strongly prefer it, because the async stuff stores a few more thing 
which would just be annoying for other types.

> BUT you can also use a functional style if you want to:
>> auto in_setting = process::std_in < null;
>> auto c = execute("thingy", in_setting);
>>
>
> I'm not suggesting that you should remove the nice API, more that it could
> be optional.
>
>
>> you have a few settings which will be aggregated like environment
>
> settings or args.
>>
>
> Yes, sorry, I didn't spot that for the env and args. However the argument
> remains for the streams initialization.

No problem, the documentation is not that detailed yet.

>
> The Problem with [2] and [3] is, that I now have the
>> initializer-sequence as a template parameter of the executor. I did this
>> so initializers can access the sequence, which was necessary for the
>> async stuff (i.e. so they can access the io_service).
>
>
> AFAIK, this was also the case in the 0.5 version
>

Nope, there execute was not a template, only the operator(). But I have 
some cross-dependency in the sequence now, due to the async stuff.

>
>> This renders any virtualization impossible, because the handler-functions
>> now have to be
>> tempalted.
>>
>
> Yes.
>
>
>> Also: though not yet implemented, I'd like to have a few compile-time
>> checks, e.g. that you don't redirect a pipe twice etc. That would be
>> completely impossible with a initializer sequence built at runtime.
>>
>
> I believe that those checks are incredibly cheap compared to a fork or a
> CreateProcess(), why not also do some runtime checks ?
>

It's not a time concern, but I like a function with invalid arguments to 
be checked at compile-time. That's the ordinary behaviour if you call a 
function with an invalid argument list. I.e. wrong at compile time -> 
error at compile time.

>
>> Now: since we have a finite amount of initializers, it would be possible
>> to implement some polymorphic sequence with boost.variant, but I still
>> fail to see why that must be determined at runtime. Keep in mind: you
>> would not determine the value of some initializer but which initializers
>> you set. Do you have an example where it has to be done at runtime? That
>> maybe helpful for me to understand the actual problem here. I really
>> don't like to use runtime-polymorphy when it can be done
>
> at compile-time
>
>
> I completely agree that compile-time checks are nice, but having a lower
> level  non fully typesafe API cannot hurt.

That is actually given by the iostreams::file_descriptor - this thing 
just wraps around a stream-handle, so I think that would be the way to 
go. You can use them for everything, but you currently have to use 
file_descriptor_sink or file_descriptor_source.

>
> An example where it is necessary to do the initialization at runtime could
> be a simple launcher that can exercise every combination of options that
> boost.process allow, like:
>
>      Usage: boost-process-launcher [OPTIONS] -- COMMAND [ARG]...
>
> That you could use like that
>
>      $ echo test | boost-process-launcher --stdout=STDERR
> --stderr=./somefile.txt --E ENVVAR=1 --no-inherit-env -- grep test
>

Ok, this I would actually implement via variants.

> Cheers,
>
> [1] https://docs.python.org/3.6/library/subprocess.html#popen-constructor
>
> _______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
>