Is there any interest in a weak pointer that plays well with shared_ptr and its friends? Wayne Conrad
At 06:54 PM 1/5/2002, wayneconrad wrote:
Is there any interest in a weak pointer that plays well with shared_ptr and its friends?
Both Greg Colvin and Peter Dimov (and others?) have worked on the idea off and on. I think they are "on" at the moment. On the main Boost list we are also looking hard at the Loki::SmartPtr. It would be interesting to try to implement cyclic_ptr, weak_ptr, and the like as a SmartPtr OwnershipPolicy. The thinking is that the future of smart pointers isn't to keep adding completely separate classes. But rather to add more instances of policy classes to a Loki::SmartPtr style framework. (For more information, see Modern C++ Design by Andrei Alexandrescu. Highly recommended!) --Beman
From: "Beman Dawes" <bdawes@acm.org>
At 06:54 PM 1/5/2002, wayneconrad wrote:
Is there any interest in a weak pointer that plays well with shared_ptr and its friends?
Both Greg Colvin and Peter Dimov (and others?) have worked on the idea off and on.
I think they are "on" at the moment.
Indeed, see http://groups.yahoo.com/group/boost/files/smart_ptr_3.zip for a first cut. -- Peter Dimov Multi Media Ltd.
--- In Boost-Users@y..., "Peter Dimov" <pdimov@m...> wrote: [...]
Indeed, see
Please revisit thread-safety/memory visibility issues wrt atomic ++/-- and MP-safe ref.counting of mutable objects... You might want to have a look at: http://groups.google.com/groups?as_umsgid=3BC8463D.6E23002A%40web.de and: http://groups.google.com/groups?as_umsgid=3C3C9B63.DFDC9920%40web.de regards, alexander.
From: "terekhov" <terekhov@de.ibm.com>
--- In Boost-Users@y..., "Peter Dimov" <pdimov@m...> wrote: [...]
Indeed, see
Please revisit thread-safety/memory visibility issues wrt atomic ++/-- and MP-safe ref.counting of mutable objects...
Could you please elaborate on that? I've read the articles at
http://groups.google.com/groups?as_umsgid=3BC8463D.6E23002A%40web.de
and:
http://groups.google.com/groups?as_umsgid=3C3C9B63.DFDC9920%40web.de
but I don't see the problem. -- Peter Dimov Multi Media Ltd.
--- In Boost-Users@y..., "Peter Dimov" <pdimov@m...> wrote:
From: "terekhov" <terekhov@d...>
--- In Boost-Users@y..., "Peter Dimov" <pdimov@m...> wrote: [...]
Indeed, see
Please revisit thread-safety/memory visibility issues wrt atomic ++/-- and MP-safe ref.counting of mutable objects...
Could you please elaborate on that? I've read the articles at
http://groups.google.com/groups?as_umsgid=3BC8463D.6E23002A% 40web.de
and:
http://groups.google.com/groups?as_umsgid=3C3C9B63.DFDC9920% 40web.de
but I don't see the problem.
Just read the following write-up: http://www.primenet.com/~jakubik/mpsafe/MultiprocessorSafe.pdf "Relaxed multiprocessor memory models...." regards, alexander.
From: "terekhov" <terekhov@de.ibm.com>
Just read the following write-up:
http://www.primenet.com/~jakubik/mpsafe/MultiprocessorSafe.pdf
"Relaxed multiprocessor memory models...."
OK, I did, but I still have trouble translating this into a specific problem with the shared_ptr code. Do you have an example? -- Peter Dimov Multi Media Ltd.
--- In Boost-Users@y..., "Peter Dimov" <pdimov@m...> wrote:
From: "terekhov" <terekhov@d...>
Just read the following write-up:
http://www.primenet.com/~jakubik/mpsafe/MultiprocessorSafe.pdf
"Relaxed multiprocessor memory models...."
OK, I did, but I still have trouble translating this into a specific problem with the shared_ptr code. Do you have an example?
Given: - a mutable shared object OBJ; - two threads THREAD1 and THREAD2 each holding a private smart_ptr object pointing to that OBJ. ---- t1: THREAD1 updates OBJ (thread-safe via some synchronization) and a few cycles later (after "unlock") destroys smart_ptr; t2: THREAD2 destroys smart_ptr WITHOUT doing any synchronization with respect to shared mutable object OBJ; OBJ destructors are called driven by smart_ptr interface... Hmm... unless you imply that smart_ptr users should treat smart_ptrs as being thread-unsafe wrt object cleanup (destructors) and should somehow always lock/unlock (synchronize) mutable shared ref.counted objects in their destructors, the bottom line is: Atomicity of --/++ is just one of two issues with respect to correctness of ref. counting in the MT environment. Visibility of eventual updates for proper destruction (object clean-up) is another issue. Currently only lock/unlock could fulfill both requirements in a portable way. regards, alexander.
From: "terekhov" <terekhov@de.ibm.com>
--- In Boost-Users@y..., "Peter Dimov" <pdimov@m...> wrote:
OK, I did, but I still have trouble translating this into a specific problem with the shared_ptr code. Do you have an example?
Given:
- a mutable shared object OBJ; - two threads THREAD1 and THREAD2 each holding a private smart_ptr object pointing to that OBJ.
----
t1: THREAD1 updates OBJ (thread-safe via some synchronization) and a few cycles later (after "unlock") destroys smart_ptr;
t2: THREAD2 destroys smart_ptr WITHOUT doing any synchronization with respect to shared mutable object OBJ; OBJ destructors are called driven by smart_ptr interface...
Yes, I see it now, thanks. I'm still not sure whether this is a shared_ptr problem... One might argue that the destructor of a mutable object is logically an "update" and therefore should be mutex-protected as the rest of the operations on OBJ unless it's guaranteed to not compete with them. On the other hand placing a memory barrier in shared_ptr will probably be a good thing from user point of view; but I don't want to make shared_ptr depend on pthreads. -- Peter Dimov Multi Media Ltd.
--- In Boost-Users@y..., "Peter Dimov" <pdimov@m...> wrote: [...]
I'm still not sure whether this is a shared_ptr problem... One might argue that the destructor of a mutable object is logically an "update" and therefore should be mutex-protected as the rest of the operations on OBJ unless it's guaranteed to not compete with them.
http://groups.google.com/groups?as_umsgid=3BCAAB99.1157302E%40web.de "> If you need to access the object to clean it up before deleting it,
you can lock it, to make sure you have the latest value. Do not assume that just because the refcount is zero, you don't have to lock.
Yeah, that would solve the visibility problem for cleanup of mutable objects. However, with a lock protecting more than one object we could still block -- which would cause an unnecessary decrease in concurrency, increased contention. Also, the memory access reordering constrain (MBAR) injected by unlock seems to be absolutely redundant to me in this situation (and we would need unlock even in the situation with a dedicated lock per object, to ensure that lock destruction would not fail with e.g. EBUSY, AFAIK)."
On the other hand placing a memory barrier in shared_ptr will probably be a good thing from user point of view;
I think so too, but how about portability... also, given rather close to NULL amount of interest (wrt the proposal for having portable interface specifically for ref.counting) I've observed on c.p.t, I am not sure at all... maybe I am missing something subtle and the whole idea of a single "RMB" might be a BS... I do not know, really.
but I don't want to make shared_ptr depend on pthreads.
Yeah, pthreads is not a boost library, but other than that argument and perhaps some licensing issues wrt LGPL nature of pthreads-win32, I do not see why boosters just ignore pthreads. It is a standard and it is available on almost every platform out there (I mean basic threading stuff). regards, alexander.
From: "terekhov" <terekhov@de.ibm.com>
On the other hand placing a memory barrier in shared_ptr will probably be a good thing from user point of view;
I think so too, but how about portability... also, given rather close to NULL amount of interest (wrt the proposal for having portable interface specifically for ref.counting) I've observed on c.p.t, I am not sure at all... maybe I am missing something subtle and the whole idea of a single "RMB" might be a BS... I do not know, really.
Perhaps C++ compilers already do the right thing on "delete p", i.e. synchronize. They need to do that anyway to protect the allocator.
but I don't want to make shared_ptr depend on pthreads.
Yeah, pthreads is not a boost library, but other than that argument and perhaps some licensing issues wrt LGPL nature of pthreads-win32, I do not see why boosters just ignore pthreads. It is a standard and it is available on almost every platform out there (I mean basic threading stuff).
pthreads is fine, it's just that the current smart_ptr.hpp doesn't need pthreads and I wanted to stay backward compatible. -- Peter Dimov Multi Media Ltd.
--- In Boost-Users@y..., "Peter Dimov" <pdimov@m...> wrote:
From: "terekhov" <terekhov@d...>
On the other hand placing a memory barrier in shared_ptr will probably be a good thing from user point of view;
I think so too, but how about portability... also, given rather close to NULL amount of interest (wrt the proposal for having portable interface specifically for ref.counting) I've observed on c.p.t, I am not sure at all... maybe I am missing something subtle and the whole idea of a single "RMB" might be a BS... I do not know, really.
Perhaps C++ compilers already do the right thing on "delete p", i.e. synchronize. They need to do that anyway to protect the allocator.
That is unlikely, I think. For example, consider that in PTHREAD memory model: "Applications shall ensure that access to any memory location by more than one thread of control (threads or processes) is restricted such that no thread of control can read or modify a memory location while another thread of control may be modifying it. Such access is restricted using functions that synchronize thread execution and also synchronize memory with respect to other threads. The following functions synchronize memory with respect to other threads: ... Applications may allow more than one thread of control to read a memory location simultaneously." It is just impossible to build correct smart_ptr that would require you to synchronize "in the destructors", even if threads unref() mutable shared objects INSIDE their synchronized regions, before "unlock" (I mean sync.regions where mutable shared objects become modified). That is just yet another reason why I dislike the idea of "synchronized destructors".
but I don't want to make shared_ptr depend on pthreads.
Yeah, pthreads is not a boost library, but other than that argument and perhaps some licensing issues wrt LGPL nature of pthreads-win32, I do not see why boosters just ignore pthreads. It is a standard and it is available on almost every platform out there (I mean basic threading stuff).
pthreads is fine, it's just that the current smart_ptr.hpp doesn't need pthreads and I wanted to stay backward compatible.
Two points: - I think that you still might want to provide a pure single-treaded version of your smart_ptr. It would be useful for pure single-threaded programs and for thread-private "shared" objects in multi- threaded programs. How about shared_ptr and thread_shared_ptr (you might also want to provide an optimized version for const shared objects without any memory sync)? BTW, it seems that currently you need TWO mutexes per counted_base for the PTHREAD version. Races aside, I think that it is really too many... - with "do not ignore PTHREADS" I just wanted to encourage you to write your code on top of PTHREAD opaque objects (pthread_mutex_t), their methods in C notation (pthread_mutex_lock) and their return codes ONLY. Just pretend that PTHREADS being the standard extension to ANSI C is available universally. Now, but what about WIN32 and users who just do not want to download pthread- win32 LGPL library, for example? NO PROBLEM. Just include your own mini-pthreads-win32 impl in your own distribution package. The same goes for any other platform you might want to support and which does not natively have PTHREADS on it. To me, it is just like some home grown C++ file streams on top of standard C fopen/fread/fwrite/etc, or in other words, just imagine that <pthread.h> will someday become <cthread> and some future C++ <thread{*}> features (btw thread_shared_ptr does belong to them ;-) will be defined using "as if" term with respect to C <cthread> features PTHREADS/*C*THREADS (they just should not forget to destroy automatic C++ objects on cancel/exit raise some C++ cancel/exit exception ;-) regards, alexander.
--- In Boost-Users@y..., "terekhov" <terekhov@d...> wrote:
--- In Boost-Users@y..., "Peter Dimov" <pdimov@m...> wrote: [...]
I'm still not sure whether this is a shared_ptr problem... One might argue that the destructor of a mutable object is logically an "update" and therefore should be mutex-protected as the rest of the operations on OBJ unless it's guaranteed to not compete with them.
http://groups.google.com/groups?as_umsgid=3BCAAB99.1157302E%40web.de
"> If you need to access the object to clean it up before deleting it,
you can lock it, to make sure you have the latest value. Do not assume that just because the refcount is zero, you don't have to lock.
Yeah, that would solve the visibility problem for cleanup of mutable objects. However, with a lock protecting more than one object we could still block -- which would cause an unnecessary decrease in concurrency, increased contention. Also, the memory access reordering constrain (MBAR) injected by unlock seems to be absolutely redundant to me in this situation (and we would need unlock even in the situation with a dedicated lock per object, to ensure that lock destruction would not fail with e.g. EBUSY, AFAIK)."
On the other hand placing a memory barrier in shared_ptr will probably be a good thing from user point of view;
I think so too, but how about portability... also, given rather close to NULL amount of interest (wrt the proposal for having portable interface specifically for ref.counting) I've observed on c.p.t, I am not sure at all... maybe I am missing something subtle and the whole idea of a single "RMB" might be a BS... I do not know, really.
I don't follow this paragraph at all. Too many acronyms I can't decipher.
but I don't want to make shared_ptr depend on pthreads.
Yeah, pthreads is not a boost library, but other than that argument and perhaps some licensing issues wrt LGPL nature of pthreads-win32, I do not see why boosters just ignore pthreads. It is a standard and it is available on almost every platform out there (I mean basic threading stuff).
A) It's not available on every platform. B) It's a C library that fails to address several C++ issues. C) It has it's own issues with many of these topics, including memory visibility. A boost solution can't address all of the above concerns, but it can come closer than POSIX and helps to define what's required when a threading library is included in the C++ standard. Bill Kempf
--- In Boost-Users@y..., "bill_kempf" <williamkempf@h...> wrote:
--- In Boost-Users@y..., "terekhov" <terekhov@d...> wrote:
--- In Boost-Users@y..., "Peter Dimov" <pdimov@m...> wrote: [...]
I'm still not sure whether this is a shared_ptr problem... One might argue that the destructor of a mutable object is logically an "update" and therefore should be mutex-protected as the rest of the operations on OBJ unless it's guaranteed to not compete with them.
http://groups.google.com/groups?as_umsgid=3BCAAB99.1157302E% 40web.de
"> If you need to access the object to clean it up before deleting it,
you can lock it, to make sure you have the latest value. Do not assume that just because the refcount is zero, you don't have to lock.
Yeah, that would solve the visibility problem for cleanup of mutable objects. However, with a lock protecting more than one object we could still block -- which would cause an unnecessary decrease in concurrency, increased contention. Also, the memory access reordering constrain (MBAR) injected by unlock seems to be absolutely redundant to me in this situation (and we would need unlock even in the situation with a dedicated lock per object, to ensure that lock destruction would not fail with e.g. EBUSY, AFAIK)."
On the other hand placing a memory barrier in shared_ptr will probably be a good thing from user point of view;
I think so too, but how about portability... also, given rather close to NULL amount of interest (wrt the proposal for having portable interface specifically for ref.counting) I've observed on c.p.t, I am not sure at all... maybe I am missing something subtle and the whole idea of a single "RMB" might be a BS... I do not know, really.
I don't follow this paragraph at all. Too many acronyms I can't decipher.
BS == BULLSHIT ;-)
but I don't want to make shared_ptr depend on pthreads.
Yeah, pthreads is not a boost library, but other than that argument and perhaps some licensing issues wrt LGPL nature of pthreads-win32, I do not see why boosters just ignore pthreads. It is a standard and it is available on almost every platform out there (I mean basic threading stuff).
A) It's not available on every platform. B) It's a C library that fails to address several C++ issues. C) It has it's own issues with many of these topics, including memory visibility.
A boost solution can't address all of the above concerns, but it can come closer than POSIX and helps to define what's required when a threading library is included in the C++ standard.
http://groups.google.com/groups?as_umsgid=3C33115D.E7EEC40%40web.de regards, alexander.
--- In Boost-Users@y..., "terekhov" <terekhov@d...> wrote:
A) It's not available on every platform. B) It's a C library that fails to address several C++ issues. C) It has it's own issues with many of these topics, including memory visibility.
A boost solution can't address all of the above concerns, but it can come closer than POSIX and helps to define what's required when a threading library is included in the C++ standard.
http://groups.google.com/groups?as_umsgid=3C33115D.E7EEC40%40web.de
A) Again, POSIX is C not C++. If the C++ standard were to adopt the POSIX pthread API and just address the concerns you mention in this post (and you missed several, including memory visibility rules that POSIX *DOESN'T* quite get correct) then that library would be usable in C++ programs, but POSIX isn't. We aren't the standard, so to use threads correctly we need our own library that addresses these issues. B) You didn't addres A in my original post. C) A C API for multi-threading is not what most people want. They want a more flexible C++ API that avoids things that are considered dangerous in C, such as void pointers. Bill Kempf
participants (5)
-
Beman Dawes -
bill_kempf -
Peter Dimov -
terekhov -
wayneconrad