Re: [boost] Going forward with Boost.SIMD

Subject: Re: [boost] Going forward with Boost.SIMD
From: Mathias Gaunard (mathias.gaunard_at_[hidden])
Date: 2013-04-23 09:43:02

On 18/04/13 16:08, Niall Douglas wrote:

> 1. GPU and CPU stream computation technologies are still merging. In other
> words, it's too soon to standardize this technology lest we accidentally
> break some novel form of new convergence. Happy to reconsider post-C++14.

I think CPUs and GPUs are different things, and that it is a mistake to 
consider an unified programming model.

A GPU is an accelerator for large regular computations, and requiring 
sending memory and receiving it back. It's also programmed with a very 
constrained programming model that cannot express efficiently all kinds 
of operations.

A CPU, on the other hand, is a very flexible processor and all memory is 
already there. You can make it do a lot of complex computations, 
irregular, sparse or iterative, can do dynamic scheduling and work 
stealing, and have fine-grained control on all components and how they 
work together.

> 2. It's hard to standardize current CPU SIMD implementations due to
> extremely irritating inconsistencies between vendors. For example, a generic
> straight port of SSE2 to NEON will have awful performance because SSE2 code
> does a lot of flipping between SIMD and non-SIMD, and because NEON is a
> coprocessor on ARM that generates poor performance. Another bug bear of mine
> on NEON is the lack of an equivalent to _mm_movemask_epi8(), which can be
> emulated in about eight NEON instructions, but in so doing you'll make code
> which was very speedy on SSE2 pretty slow in most cases on NEON.
>
> My point here is you cannot standardize such non-uniform behavior in a
> universally performant way, because you'll just get lowest common
> denominator performance across all SIMD implementations which kinda defeats
> its purpose. If all vendors were like NEON, or like SSE2, then we still have
> to see how CUDA and OpenCL pan out long run.

A developer should try to write code that is as vertical as possible 
(i.e. that does the same operation on all elements of the register, and 
doesn't mix between the elements).
Doing this will give performance on all kinds of SIMD hardware. Using a 
couple of regular horizontal operations might be ok, but if your 
algorithm is heavily dependent on more advanced operations like 
shuffling, performance will indeed be hardware-dependent.

The main use case of _mm_movemask_epi8 is to implement reduction 
functions (which are horizontal) like all, none or any. NEON does not 
have an instruction to do this directly, but you can do it with 
relatively few instructions using VHADD; while it might not be as fast 
as SSE it shouldn't be too bad either.

> 3. I am unsure if C++ is the appropriate language for SIMD standardization
> when perhaps a meta-form of JIT compiled C++ would be much superior (i.e.
> you supply LLVM bytecode, and it gets delivered to a GPU/CPU/whatever).
> We'll have those on the table with LLVM-type compilers. In other words, I
> would vote to wait and see what the market throws up.

Hardware manufacturers already provide a C programming interface to use 
them.
The idea of a C++ library rather than C is to be able to use the native 
operators and function overloading.

> I appreciate that none of these three rationale are what you want to hear.
> Still, I hope my observations are useful to you. None of them suggests you
> shouldn't proceed with Boost.SIMD. Boost has a much wider remit than just as
> a testing ground for future C++ standard library features. But I suspect
> that if SIMD ever does get standardized, it won't look like your library or
> proposal because it will be based on technologies which don't exist yet.
>
> Hope that helps,

This is similar to the feedback I received at the meeting in Bristol.

However, SIMD has been here for 25 years and is still in the roadmap of 
future processors. Across all this time it has mostly stayed the same.

On the other hand GPU computing is relatively new and is evolving a lot. 
It's also quite trendy and buzzword-y, and is in reality not as fast and 
versatile as marketing makes it out to be.
A lot of people seem to be intent on standardizing GPU technology rather 
than SIMD technology; that's quite a shame.