Re: [Boost-users] Subject:[atomic] why atomic<>::store method can not support memory_order_consum memory_order_acquire and memory_order_acq_rel these memory_orders?

Subject: Re: [Boost-users] Subject:[atomic] why atomic<>::store method can not support memory_order_consum memory_order_acquire and memory_order_acq_rel these memory_orders?
From: class7class_at_[hidden]
Date: 2015-07-23 03:25:27

Your explanation is very detailedand useful, I think I understood.

Thank you very much!





class7class_at_[hidden]
 
From: boost-users-request
Date: 2015-07-23 09:04
To: boost-users
Subject: Boost-users Digest, Vol 4201, Issue 4
Send Boost-users mailing list submissions to
boost-users_at_[hidden]
 
To subscribe or unsubscribe via the World Wide Web, visit
http://lists.boost.org/mailman/listinfo.cgi/boost-users

or, via email, send a message with subject or body 'help' to
boost-users-request_at_[hidden]
 
You can reach the person managing the list at
boost-users-owner_at_[hidden]
 
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Boost-users digest..."
 
 
Today's Topics:
 
   1. Re: Boost release 1.59.0 beta 1 is now available from
      SourceForge (Ben Pope)
   2. Re: Boost release 1.59.0 beta 1 is now available from
      SourceForge (Vicky Vergara)
   3. Re: Subject:[atomic] why atomic<>::store method can not
      support memory_order_consum memory_order_acquire and
      memory_order_acq_rel these memory_orders? (Gavin Lambert)
   4. [filesystem] problem: is_regular_file and deduplified files
      (reparse+sparse) (Paul Harris)
   5. Re: Subject:[atomic] why atomic<>::store method can not
      support memory_order_consum memory_order_acquire and
      memory_order_acq_rel (Isaac To)
 
 
----------------------------------------------------------------------
 
Message: 1
Date: Thu, 23 Jul 2015 05:20:39 +0800
From: Ben Pope <benpope81_at_[hidden]>
To: boost-users_at_[hidden]
Cc: boost_at_[hidden]
Subject: Re: [Boost-users] Boost release 1.59.0 beta 1 is now
available from SourceForge
Message-ID: <mop1f7$6dg$1_at_[hidden]>
Content-Type: text/plain; charset=windows-1252; format=flowed
 
On Thursday, July 23, 2015 02:23 AM, Marshall Clow wrote:
> On Thu, Jul 16, 2015 at 6:16 AM, Marshall Clow <mclow.lists_at_[hidden]
> <mailto:mclow.lists_at_[hidden]>> wrote:
>
>
>     Boost release 1.59.0 beta 1 is now available from SourceForge
>     See http://sourceforge.net/projects/boost/files/boost/1.59.0.beta.1/

>     For details of what's in the release, see
>     http://www.boost.org/users/history/version_1_59_0.html.

>     Note that the links to files on this web page are for the final
>     release - use the SourceForge link above to get the beta files.
>     Please download the beta, give it a try, and report any problems you
>     encounter.
>
>
> Has anyone tried the beta release with new compilers?
>
> Specifically, I'm thinking about:
> * GCC 5.2
> * VS2015
> * Clang 3.7rc1
 
For clang 3.7 rc1, the tests for master are up on the ftp, but they 
won't be published because I haven't submitted a pull request for them 
to be published on the test site, since I don't run them regularly.
 
The dev results should be up on the test matrix at some point soon, but 
the entire run of dev and release takes a couple of days (which feels 
slow) and is still in progress.
 
I intend to publish the results of asan, msan, tsan runs at some point 
but I haven't got round to figuring out the right way to present them 
yet, if you want the raw xml results and don't have access to the ftp, 
contact me.
 
I'll probably run for gcc 5.2 when it arrives on Ubuntu's launchpad.
 
Ben
 
 
 
 
------------------------------
 
Message: 2
Date: Wed, 22 Jul 2015 17:53:17 -0500
From: Vicky Vergara <vicky_vergara_at_[hidden]>
To: "boost-users_at_[hidden]" <boost-users_at_[hidden]>
Subject: Re: [Boost-users] Boost release 1.59.0 beta 1 is now
available from SourceForge
Message-ID: <BAY177-W30049CC41A57EE28144AA18A830_at_phx.gbl>
Content-Type: text/plain; charset="iso-8859-1"
 
We re working on the release of pgRouting. We use  Boost.Graph.
My Question is:
Waht changes I have to do in Github travis to check that pgRouting is compatible with this beta release?
 
Here is our travis file:
https://github.com/pgRouting/pgrouting/blob/develop_2_1_0/.travis.yml

 
Right now it runs with boost 1.46.1
https://travis-ci.org/pgRouting/pgrouting/jobs/72064827#L1834

 
Thanks
 
> To: boost-users_at_[hidden]
> From: benpope81_at_[hidden]
> Date: Thu, 23 Jul 2015 05:20:39 +0800
> CC: boost_at_[hidden]
> Subject: Re: [Boost-users] Boost release 1.59.0 beta 1 is now available from SourceForge
> 
> On Thursday, July 23, 2015 02:23 AM, Marshall Clow wrote:
> > On Thu, Jul 16, 2015 at 6:16 AM, Marshall Clow <mclow.lists_at_[hidden]
> > <mailto:mclow.lists_at_[hidden]>> wrote:
> >
> >
> >     Boost release 1.59.0 beta 1 is now available from SourceForge
> >     See http://sourceforge.net/projects/boost/files/boost/1.59.0.beta.1/

> >     For details of what's in the release, see
> >     http://www.boost.org/users/history/version_1_59_0.html.

> >     Note that the links to files on this web page are for the final
> >     release - use the SourceForge link above to get the beta files.
> >     Please download the beta, give it a try, and report any problems you
> >     encounter.
> >
> >
> > Has anyone tried the beta release with new compilers?
> >
> > Specifically, I'm thinking about:
> > * GCC 5.2
> > * VS2015
> > * Clang 3.7rc1
> 
> For clang 3.7 rc1, the tests for master are up on the ftp, but they 
> won't be published because I haven't submitted a pull request for them 
> to be published on the test site, since I don't run them regularly.
> 
> The dev results should be up on the test matrix at some point soon, but 
> the entire run of dev and release takes a couple of days (which feels 
> slow) and is still in progress.
> 
> I intend to publish the results of asan, msan, tsan runs at some point 
> but I haven't got round to figuring out the right way to present them 
> yet, if you want the raw xml results and don't have access to the ftp, 
> contact me.
> 
> I'll probably run for gcc 5.2 when it arrives on Ubuntu's launchpad.
> 
> Ben
> 
> 
> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden]
> http://lists.boost.org/mailman/listinfo.cgi/boost-users

     
-------------- next part --------------
HTML attachment scrubbed and removed
 
------------------------------
 
Message: 3
Date: Thu, 23 Jul 2015 12:20:28 +1200
From: Gavin Lambert <gavinl_at_[hidden]>
To: boost-users_at_[hidden]
Subject: Re: [Boost-users] Subject:[atomic] why atomic<>::store method
can not support memory_order_consum memory_order_acquire and
memory_order_acq_rel these memory_orders?
Message-ID: <mopc0e$6ph$1_at_[hidden]>
Content-Type: text/plain; charset=windows-1252; format=flowed
 
On 22/07/2015 14:43, class7class_at_[hidden] wrote:
> I am a SE and I program based on boost library, I have one question to
> ask as below:
>
> When I use the atomic library included boost , I coded as following:
>    atomic<bool> flag; //declare the var
>                    flag.store(true,boost::memory_order_acq_rel); // use
> store method to change the value of flag
>
> but BOOST_ASSERT failed,and I read the code of base_atomic<> template
> class,I found the the store() method can not support several
> memory_orders metioned in the subject , and the other methods can not
> support all memory_orders,such as:
>    1.the load method can not support memory_order_release and
> memory_order_acq_rel
>    2. the compare_exchange_strong method can not support
> memory_order_release and memory_order_acq_rel
>    3. the compare_exchange_weak method can not support
> memory_order_release and memory_order_acq_rel
>
> I really want to know the reason that why the methods of atomic can not
> support all memory_orders.
 
Because this is mandated by the standard, and because doing otherwise 
doesn't really make sense.
 
Loads can't release, and stores can't acquire.  Therefore those memory 
orders are nonsensical.
 
memory_order_acq_rel exists for RMW operations such as compare_exchange. 
  The "acq" applies to the load and the "rel" to the store.  It's not 
sensible for a store alone.
 
> If I need the methods of the atomic<> support all memory_orders,how
> can i do?
 
You can re-read the standard memory model and discover why that doesn't 
make sense.
 
Memory ordering can be tricky to get your head around at first -- it's a 
good idea to run any atomic algorithm you implement through a sanity 
checker such as RRD or ThreadSanitizer to help catch subtle bugs.
 
If in doubt, stick with seq_cst memory ordering (the default).  On 
x86/x64 there is very little performance difference between seq_cst and 
the weaker orders -- cacheline sharing tends to be more problematic.
 
 
 
 
------------------------------
 
Message: 4
Date: Thu, 23 Jul 2015 08:56:21 +0800
From: Paul Harris <harris.pc_at_[hidden]>
To: boost-users <boost-users_at_[hidden]>
Subject: [Boost-users] [filesystem] problem: is_regular_file and
deduplified files (reparse+sparse)
Message-ID:
<CAMSUDdZL95bnn3Ec=3pKGkKRdtMk2i7=PL6e7aK6u4J9GBVpZg_at_[hidden]>
Content-Type: text/plain; charset="utf-8"
 
Hi guys,
 
I'm seeing a lot of failures all of a sudden, its a serious problem.  I
imagine some of you may start seeing the same failures too very soon.
 
I'm seeing it with boost 1.58.0.
There is a related bug already entered here:
https://svn.boost.org/trac/boost/ticket/11057

I can't seem to log on and update this ticket.
 
But the problem extends further than that ticket.
 
Windows Server 2003 support has been dropped by Microsoft, and a lot of IT
depts around the world are switching over to the latest Windows Server.
 
With this server comes the new "dedup" feature, that can automatically
deduplify files.  This happens on a schedule, eg 2am saturday.  So suddenly
we are getting messages of failures of software from all over the place,
due to fs::is_regular_file()
 
Deduped files have the REPARSE and SPARSE flag set.
On the command line, you can run
FSUTIL REPARSEPOINT QUERY
 
and the "Reparse Tag Value" is 0x80000013
 
Which is a relatively new flag known as IO_REPARSE_TAG_DEDUP
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365740%28v=vs.85%29.aspx

 
These files act as normal files, you can fopen and fread them, so I assume
they should be treated almost like symlink by boost... perhaps not quite a
symlink because I assume the "lstat" link properties are identical to the
file's stat properties.
 
 
Typically, I iterate over directories and only process files if
fs::is_regular_file(filename) is true.
 
I wrote some code to check what the properties were on these files, and its
not any of the possible enums detected by file_status::type().
 
ideas?
 
Best regards,
Paul
-------------- next part --------------
HTML attachment scrubbed and removed
 
------------------------------
 
Message: 5
Date: Thu, 23 Jul 2015 09:04:57 +0800
From: Isaac To <isaac.to_at_[hidden]>
To: boost-users_at_[hidden]
Subject: Re: [Boost-users] Subject:[atomic] why atomic<>::store method
can not support memory_order_consum memory_order_acquire and
memory_order_acq_rel
Message-ID:
<CAOArY3UmCXoCcWPntHo2GufdmAy1mSNceFGUwdoZt4_Kd5D2EA_at_[hidden]>
Content-Type: text/plain; charset="utf-8"
 
On Thu, Jul 23, 2015 at 12:00 AM, "class7class_at_[hidden]" <class7class_at_[hidden]
> wrote:
 
> I found the the store() method can not support several memory_orders
> metioned in the subject , and the other methods can not support all
> memory_orders,such as:
>   1.the load method can not support memory_order_release and
> memory_order_acq_rel
>   2. the compare_exchange_strong method can not support
> memory_order_release and memory_order_acq_rel
>   3. the compare_exchange_weak method can not support memory_order_release
> and memory_order_acq_rel
>
> I really want to know the reason that why the methods of atomic can not
> support all memory_orders.
>
> If I need  the methods of the atomic<> support all memory_orders,how can i
> do?
>
 
You are asking into the area of C++ where the fewest people in the world
understand, namely relaxed atomics.  Before I tell my limited understanding
of the issue, I have an advice:
 
   - Use mutexes like boost::mutex to ensure all your accesses to variables
   shared among threads are not accessed concurrently, and you'll be fine.
 
If your circumstance forces you to ignore the above advice (like me, e.g.,
when you cannot afford the scheduling overheads of mutexes), here is yet
another advice which may save your life:
 
   - Make all your variables that can be accessed concurrently from
   multiple threads (with at least one thread writing) atomic.  (This makes
   your program "data race free".)
   - Always use the default memory_order_seq_cst memory ordering for
   accesses to your atomics, never use anything else.
 
If your circumstance forces you to ignore the above advices (unlike me, I
just learn them because I haven't heard about the second advice before;
you'll have to be writing for old ARM (i.e., ones without the LDA and STL
instructions) or POWER for you to have a real reason to use relaxed
atomics, I write for neither), here is what I have learnt.
 
tl;dr
 
The boost atomic doesn't support all memory ordering because the C++11
standard doesn't support all of them.  The C++ standard doesn't support all
memory ordering because not all make sense in the "memory model" supported
by C++.  It is not likely to change in the near future, i.e., unless some
genius devises some novel memory model which changes the status quo.
 
Here "memory model" means what "loads" from memory is allowed to return.
Because of compiler and hardware optimization, writes (including atomic
writes) to memory locations may be reordered, so the load doesn't always
return "the last value stored".  Special hardware instructions are
necessary to ensure a particular ordering that the programmer needs.  The
easiest way to understand a program is that "All memory operations by all
threads looks as if they are done one after another, interleaving among
threads".  But it proves to be too tricky for hardware to provide any
reasonable performance for this model.
 
The C++ memory model without relaxed atomics is essentially "All *atomic*
memory operations by all threads looks as if they are done one after
another, interleaving among threads.  All other memory operations of each
thread looks as if they are completely done between adjacent atomic memory
operations.  But the programmer guarantees not to write a data race.  The
compiler uses this assumption in all its work, and your program can break
in all mysterious ways if you break this rule."  This is essentially the
illusion provided by sequential consistency.  Everybody should want it.
 
Except some don't, because in some architectures it is essentially
impossible to provide sequential consistency efficiently.  To do sequential
consistency, one has to ensure that (1) the compiler doesn't reorder the
statements written by the programmer, which is controllable by compiler
writer; and (2) the hardware doesn't reorder the instructions generated by
the compiler, which is damn hard.  It usually means the compiler must
insert memory fences before and after memory accesses (i.e., tell the CPU
"don't return to me until everything I do up to this point are visible by
all other CPUs" or "don't return to me until everything other CPUs do up to
this point are visible by me"), which is very slow.  It is particularly bad
because for these architectures, fences must be inserted not just for
atomic stores, but for the supposedly fast atomic loads as well.  (Recent
architectures do better because they link the requirement into the accesses
of the particular memory location: the "particular memory location" here
limits the scope where the "fences" have to do, and this makes a big
difference in performance.)
 
That's why relaxed atomics (i.e., memory_order_acq, etc) exists in the
standard.  They allow those architectures to perform better than using full
sequential consistency in some common cases.  Because different
architectures require different sort of fences, it doesn't make sense for
these to mean "insert a fence after the load".  Instead, the C++ standard
uses a more programmer centric view when defining relaxed memory ordering:
 
   - If a thread X stores a value with release semantics into an atomic
   variable, and another thread Y loads this value with acquire semantics from
   that same atomic variable, then everything done by thread X before this
   releasing store are guaranteed to be visible to thread Y after this
   acquiring load.
 
There is no other semantics attached to the acquire and release.  This is
why acquiring stores and releasing loads are meaningless: they are not
assigned any semantics.  But the deeper reason for the lack is that the
computer industry cannot find a way to give additional guarantee without
incurring hefty performance overheads.  And at the same time, sequentially
consistent atomics are not so expensive in the new architectures.
 
As for what to do if you want more than the above release-acquire
guarantee: Simple, just use sequentially consistent atomic operations.
Regards,
Isaac
-------------- next part --------------
HTML attachment scrubbed and removed
 
------------------------------
 
Subject: Digest Footer
 
_______________________________________________
Boost-users mailing list
Boost-users_at_[hidden]
http://lists.boost.org/mailman/listinfo.cgi/boost-users

 
------------------------------
 
End of Boost-users Digest, Vol 4201, Issue 4
********************************************