Re: [boost] GSOC 2015 : Project on Concurrent Hash Tables

Subject: Re: [boost] GSOC 2015 : Project on Concurrent Hash Tables
From: Niall Douglas (s_sourceforge_at_[hidden])
Date: 2015-03-05 23:19:34

On 6 Mar 2015 at 6:13, Amarnath V A wrote:

> 1. After the discussion with Niall and Rob on the piece of code I have
> shared here before, what I understood is that I was not marking the
> old map to be reloaded by the threads which are making use of it. Say,
> I performed a move construction of old_map and it was being used by
> few threads, shouldn't I mark it for reloading by those threads. From
> the explanation Niall had provided here, I came to the conclusion that
> I have to mark the newly created buckets of old_map as lock state 2.
> But this did not work out as expected. I ended up in a infinite loop I
> guess when I ran a simple unit test testing the move construction.

You had the situation inverted. New buckets are normally locked (1) 
until the copy/move is complete. Old buckets, shortly to be emptied, 
are locked to mark death (2) and stay that way forever.

> Niall, after marking the buckets to be reloaded, how is the actual
> reload of the buckets performed? Do I have to some how let the threads
> know that you have to do a reload of the buckets? How is this
> achieved? I see that the lock stays in state 2 after I set it and is
> not switching back to 0 value.

2 = this bucket is dead.

> 2. For copy construction, should I do similar logic as in _rehash()
> method or can I just perform an atomic load and store from the
> old_map's buckets? I believe this is not the right way and should do
> something similar to the _rehash().

Some would say that a copy should be a complete atomic snapshot of 
the original. Others would say that if concurrent modification is 
occurring, you get a "best effort" copy. I leave that decision to the 
student, so long as it's thread safe I don't mind.

> 3. How do I handle exceptions in move construction? On an exception,
> what is the move construction expected to do? Just abort whatever it
> was doing and restore the old_map?

Exception safety means restoring the map to as if no change had 
happened with no resources leaked or data changed.

> 4. And how about copy constructor? Are copy constructors allowed to
> throw exceptions?

Of course. Just release any memory allocated, and throw the exception 
back up to the caller.

> 5. Niall, one  more question. I see that the thread_sanitizer unit
> test is not building. Please see the build #228's console output.
> https://ci.nedprod.com/view/Boost%20Thread-Expected-Permit/job/Boost.Spinlock%20Test%20Linux%20GCC%204.8/lastCompletedBuild/console
> 
> I see that there is an unrecognized flag "-fsanitize=undefined" and
> compiler is throwing the following error.
> 
> g++: error: unrecognized command line option '-fsanitize=undefined'

That was my sloppiness on the CI config, sorry, I was running the 
wrong version of GCC. Fixed now.

> I removed the particular flag from the build script and compiled. This
> time it worked out and but when I ran, I see so many warnings thrown.
> Is this expected or is it that something needs to be fixed?

You need to set the TSAN_OPTIONS environment variable to the 
suppressions file like this:

export TSAN_OPTIONS="suppressions=tsan.supp history_size=7"

Obviously adjusting the path appropriately. You then run the 
unittests_sanitise binary. The unit tests should execute very slowly, 
but hopefully no errors appear.

Try it with an unmodified version first before trying your own. And 
let me know if it fails, the fact the CI wasn't testing it in some 
months means a race could have crept in. I'm currently on the wrong 
computer to test that for myself, but I will do so tomorrow.

> And I will post any other problems faced from now on. I was under the
> impression that as this is part of competency test, the students are
> supposed to figure out how to do themselves. Thanks for helping out.

Students are supposed to behave as any Boost engineer would. That 
includes asking for help and advice. Including on stackoverflow if 
you think it would help, though remember to follow stackoverflow's 
guidelines for questions. Do let us know if you decide to post there 
so I can see what replies people make.

Niall

-- 
ned Productions Limited Consulting
http://www.nedproductions.biz/ 
http://ie.linkedin.com/in/nialldouglas/