Re: [boost] Futures (was: Re: [compute] Some remarks)

Subject: Re: [boost] Futures (was: Re: [compute] Some remarks)
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2015-01-06 05:21:34

On 5 Jan 2015 at 7:52, Hartmut Kaiser wrote:

> > I don't think it's that easy because really it comes down to
> > commonality of kernel wait object, or rather, whether one has access
> > to the true underlying kernel wait object or not.
> 
> Relying on the kernel to do threading is plain slow - at least with what we
> usually have today. 

It's also often the only way to do parallelism in the kernel if yours 
is not microkernel, so if you need that then you're stuck. File i/o 
in particular (or rather, the kernel file page cache) is often 
optimised for kernel threading.

Also as I mentioned in the other email to Thomas it's usually the 
case that all waits need to have the potential to go to a kernel 
wait, even if rarely used.

> > For example, right now can boost::wait_all() ever consume
> > std::futures? I suspect not because the HANDLE on Windows or the
> > futex on Linux is rather hard to get at.
> 
> You don't need to - as long as the interfaces exposed by that concept are
> powerful enough to handle things, that is.

With C code as well?

A large chunk of newly written systems code is still in C. And I 
don't expect that to change. C++ already spends too much time in its 
compile time ivory tower and not attending to its main use case, 
systems programming.

> > Far, far better to fix the present future promise API to stop
> > requiring shared state and therefore memory allocation at all.
> 
> Not sure you can implement a shared_future without a shared state.

shared_future is a particularly broken design. What should have been 
implemented is this:

promise<T, E> -> consuming_future<T, E>. One to one correspondance, 
destructive get().

promise<T,E> -> nonconsuming_future<T, E>. One to one correspondance, 
non-destructive get().

template<class T, class E> using shared_future = 
shared_ptr<nonconsuming_future<T, E>;

This way if you need multiple futures all pointing to the same value 
with non-destructive get(), you pay for that with an explicit use of 
shared_ptr.

It's too late to fix now, but my intended basic_promise and 
basic_future works like the above. Its shared_future is implemented 
as an API thunk wrapper around a shared_ptr<nonconsuming_future<T, 
E>> so it walks and quacks like a std::shared_future.

> Sure, let's do that. It will make futures more efficient, however does not
> solve you initial concern of 'future islands' as it introduces yet another
> future type into the mix.

Correct. My proposal is a "future island factory".

> > Eliminating future islands is, I suspect, not something the C++
> > community can entirely do alone. We are, as a minimum, going to have
> > to petition POSIX for improved runtime support. We probably ought to
> > have our ducks in a row before that though.
> 
> *shiver*
> 
> I'd rather embrace future islands as we will not be able to solve this.
> Let's use the facilities available to us (C++!) and solve it inside the
> language/library.

POSIX is modifiable. We basically need a use case to persuade the 
libc maintainers to incorporate the pthreads permit object into their 
libc's, and after that the AWG cannot prevent its eventual 
standardisation.

Just to be clear, the pthreads permit object would be the absolutely 
lowest layer object. Most threading implementations in C++ ought to 
never need to reach that object most of the time.

> > I am personally surprised that Chris hasn't proposed this yet in one of
> > this N-papers proposing the ASIO way of doing async instead of the current
> > approach by the committee :)
> 
> Again, I don't see the 'current way' and 'Chris' way' as contradicting. They
> are orthogonal and address different things.

As a way of sending values between threads, yes they are orthogonal.

As a way of end user code getting a handle to some async operation it 
can poll or wait upon, they are commensurate, and Chris' way has the 
big advantage of already being standard practice in ASIO, being 
portable, and being available now and not in 2019 or later. Most C++ 
programmers just want something which solves future islands or isn't 
crazy inefficient for small grained operations right now, and Chris' 
way delivers that right now.

I can't speak for Chris, but his N-papers to WG21 read to me as him 
effectively saying that the approach currently taken by the 
Concurrency TS is fundamentally unwise given that the battle tested 
ASIO approach is already standard practice, doesn't require magic 
compiler support, is known to be highly flexible and efficient and as 
ASIO is going to become the Networking TS, it is implied is a 
superior approach for concurrency in C++.

For the record, I don't agree with that assessment, but he makes the 
especially good point that the present Concurrency TS is currently 
appearing to ignore the exigencies imposed by the likely Networking 
TS. The two idioms need to be reconciled properly into something 
which hangs well together, else we're going to see a dog's breakfast 
of concurrency support in C++ with threading and networking taking 
dimorphic concurrency paradigms. Obviously Hartmut you're on the 
appropriate committees and so are privy to details I am not, so it 
may be you are already working on this reconciliation, if so I 
applaud it.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ 
http://ie.linkedin.com/in/nialldouglas/