Re: [boost] [optional] operator<(optional<T>, T) -- is it wrong?

Subject: Re: [boost] [optional] operator<(optional<T>, T) -- is it wrong?
From: Howard Hinnant (howard.hinnant_at_[hidden])
Date: 2014-12-03 11:24:07

On Dec 2, 2014, at 7:31 PM, Niall Douglas <s_sourceforge_at_[hidden]> wrote:

> On 2 Dec 2014 at 17:40, Howard Hinnant wrote:
> 
>>> Out of curiosity I went and looked up what Python did to solve this 
>>> (here's the discussion: http://bugs.python.org/issue13703). In the 
>>> end, they simply XOR salted the hash with a cryptographically 
>>> generated random number produced at process launch (source: 
>>> https://docs.python.org/3/reference/datamodel.html#object.__hash__) 
>>> and carried on with their previous algorithm.
>> 
>> Here is a very short python script:
>> 
>> https://131002.net/siphash/poc.py
>> 
>> which appears to recover those random salts for the process.  I tried it
>> out on 2.7.5 and it appears to work really well.
> 
> This is as expected, as Pythons before 3.3 require a special 
> environment variable to be set to turn on the secure hash. Apparently 
> many Python programs assume the order of the standard hash algorithm, 
> so they couldn't just turn it on without years of warning about the 
> change.
> 
> I just tried your script on Python 3.4 and got 0 candidate solutions 
> whereas 2.7 does yield solutions. I don't claim their chosen solution 
> is foolproof, but Python probably does see a lot more untrusted 
> inputs than probably C++ does. If there were a gaping security hole 
> there, we would surely have heard about it.

More info on why this attack failed on Python 3.4:

http://lwn.net/Articles/574761/
https://www.python.org/dev/peps/pep-0456/

These articles claim that Python 3.4 and forward have simply adopted SipHash.

For anyone interested, there is a hash_append-compatible implementation of the SipHash24 variant here, in the files siphash.h/siphash.cpp:

https://github.com/HowardHinnant/hash_append

You can experiment with this code without getting into hash_append by simply doing:

#include <iostream>
#include <random>
#include <string>
#include "siphash.h"

int
main()
{
    // initialize random seeding
    std::random_device rd;
    std::mt19937_64 eng{rd()};

    // initialize siphash object
    acme::siphash hash(eng(), eng());

    // get something to hash
    std::string s("some data”);

    // hash it
    hash(s.data(), s.size());
    auto key = static_cast<std::size_t>(hash);

    // output the hash
    std::cout << key << '\n';
}

Howard