Boost logo

Boost :

Subject: Re: [boost] Going forward with Boost.SIMD
From: Mathias Gaunard (mathias.gaunard_at_[hidden])
Date: 2013-04-23 09:43:02

On 18/04/13 16:08, Niall Douglas wrote:

> 1. GPU and CPU stream computation technologies are still merging. In other
> words, it's too soon to standardize this technology lest we accidentally
> break some novel form of new convergence. Happy to reconsider post-C++14.

I think CPUs and GPUs are different things, and that it is a mistake to
consider an unified programming model.

A GPU is an accelerator for large regular computations, and requiring
sending memory and receiving it back. It's also programmed with a very
constrained programming model that cannot express efficiently all kinds
of operations.

A CPU, on the other hand, is a very flexible processor and all memory is
already there. You can make it do a lot of complex computations,
irregular, sparse or iterative, can do dynamic scheduling and work
stealing, and have fine-grained control on all components and how they
work together.

> 2. It's hard to standardize current CPU SIMD implementations due to
> extremely irritating inconsistencies between vendors. For example, a generic
> straight port of SSE2 to NEON will have awful performance because SSE2 code
> does a lot of flipping between SIMD and non-SIMD, and because NEON is a
> coprocessor on ARM that generates poor performance. Another bug bear of mine
> on NEON is the lack of an equivalent to _mm_movemask_epi8(), which can be
> emulated in about eight NEON instructions, but in so doing you'll make code
> which was very speedy on SSE2 pretty slow in most cases on NEON.
> My point here is you cannot standardize such non-uniform behavior in a
> universally performant way, because you'll just get lowest common
> denominator performance across all SIMD implementations which kinda defeats
> its purpose. If all vendors were like NEON, or like SSE2, then we still have
> to see how CUDA and OpenCL pan out long run.

A developer should try to write code that is as vertical as possible
(i.e. that does the same operation on all elements of the register, and
doesn't mix between the elements).
Doing this will give performance on all kinds of SIMD hardware. Using a
couple of regular horizontal operations might be ok, but if your
algorithm is heavily dependent on more advanced operations like
shuffling, performance will indeed be hardware-dependent.

The main use case of _mm_movemask_epi8 is to implement reduction
functions (which are horizontal) like all, none or any. NEON does not
have an instruction to do this directly, but you can do it with
relatively few instructions using VHADD; while it might not be as fast
as SSE it shouldn't be too bad either.

> 3. I am unsure if C++ is the appropriate language for SIMD standardization
> when perhaps a meta-form of JIT compiled C++ would be much superior (i.e.
> you supply LLVM bytecode, and it gets delivered to a GPU/CPU/whatever).
> We'll have those on the table with LLVM-type compilers. In other words, I
> would vote to wait and see what the market throws up.

Hardware manufacturers already provide a C programming interface to use
The idea of a C++ library rather than C is to be able to use the native
operators and function overloading.

> I appreciate that none of these three rationale are what you want to hear.
> Still, I hope my observations are useful to you. None of them suggests you
> shouldn't proceed with Boost.SIMD. Boost has a much wider remit than just as
> a testing ground for future C++ standard library features. But I suspect
> that if SIMD ever does get standardized, it won't look like your library or
> proposal because it will be based on technologies which don't exist yet.
> Hope that helps,

This is similar to the feedback I received at the meeting in Bristol.

However, SIMD has been here for 25 years and is still in the roadmap of
future processors. Across all this time it has mostly stayed the same.

On the other hand GPU computing is relatively new and is evolving a lot.
It's also quite trendy and buzzword-y, and is in reality not as fast and
versatile as marketing makes it out to be.
A lot of people seem to be intent on standardizing GPU technology rather
than SIMD technology; that's quite a shame.

Boost list run by bdawes at, gregod at, cpdaniel at, john at