Re: [boost] Any interest in hashing algorithms SHA and/or FNV1a?

Subject: Re: [boost] Any interest in hashing algorithms SHA and/or FNV1a?
From: Jeff Flinn (Jeffrey.Flinn_at_[hidden])
Date: 2013-11-13 12:48:35

On 11/13/2013 12:11 PM, Niall Douglas wrote:
> On 12 Nov 2013 at 10:52, foster brereton wrote:
>
>> Would these algorithms be of general use to the Boost community? If so
>> I would be willing to submit them formally.
>
> Be aware that AFIO (currently in the peer review queue) is slowly
> gaining an asynchronous batch hash engine. It's a bit different from
> your normal hash engine, because it can do things like use SIMD to
> process four SHA256 streams in parallel, and then using AFIO's
> closure engine to parallelise that 4-SHA256 processing across
> multiple cores i.e. achieve sixteen parallel SHA256 processing
> streams. This lets one drop the normal 14.9 cycles/byte down to
> around 1.4 cycles/byte amortised for SHA256 [1], a big win. The batch
> hash engine uses a compile-time plugin system, so it can be
> arbitrarily extended with other hash implementations if suitably
> rewritten to fit.
>
> Of course AFIO may not pass peer review, so any work done here may be
> moot. That therefore should not inhibit your efforts.
>
> In case anyone wants to know "when will it be ready?", well being
> unemployed and having to relocate back to Europe has punched a huge
> hole in my productivity. Once I'm back (January) I would expect
> things to move far more swiftly. I should have something which passes
> basic unit tests this month though - I already have a proof of
> concept working, it's just converting it into something not so hacked
> together.
>
> [1] These figures are for an Intel i7-3770K running x64. Other CPUs
> vary hugely (e.g. ARM, whose NEON unit is not well scheduled by GCC
> or clang).

0Sounds interesting. Is the hash engine customizable to composite 
algorithms? Particularly MD5/SHA1?

I'm blocked from using AFIO(on Windows) for my file processing as I need 
to go thru BackupRead/BackupWrite api's which are explicitly 
incompatible with overlapped io. I haven't looked if AFIO could offer 
benefits with that restriction.

Thanks, Jeff