[Boost-commit] svn:boost r79898 - in trunk/libs/interprocess: doc proj/vc7ide test

Subject: [Boost-commit] svn:boost r79898 - in trunk/libs/interprocess: doc proj/vc7ide test
From: igaztanaga_at_[hidden]
Date: 2012-08-07 05:13:24

Author: igaztanaga
Date: 2012-08-07 05:13:23 EDT (Tue, 07 Aug 2012)
New Revision: 79898
URL: http://svn.boost.org/trac/boost/changeset/79898

Log:
Reimplemented `message_queue` with a circular buffer index
Text files modified: 
   trunk/libs/interprocess/doc/interprocess.qbk             |    64 +++++++++++++++++++++++++++++++++++-----
   trunk/libs/interprocess/proj/vc7ide/message_queue.vcproj |     8 ++--                                    
   trunk/libs/interprocess/test/message_queue_test.cpp      |    35 ++++++++++++++++++---                   
   3 files changed, 90 insertions(+), 17 deletions(-)

Modified: trunk/libs/interprocess/doc/interprocess.qbk
==============================================================================
--- trunk/libs/interprocess/doc/interprocess.qbk	(original)
+++ trunk/libs/interprocess/doc/interprocess.qbk	2012-08-07 05:13:23 EDT (Tue, 07 Aug 2012)
@@ -622,14 +622,9 @@
 Windows shared memory creation is a bit different from portable shared memory
 creation: the size of the segment must be specified when creating the object and
 can't be specified through `truncate` like with the shared memory object.
-
 Take in care that when the last process attached to a shared memory is destroyed
 [*the shared memory is destroyed] so there is [*no persistency] with native windows
-shared memory. Native windows shared memory has also another limitation: a process can
-open and map the whole shared memory created by another process but it can't know
-which is the size of that memory. This limitation is imposed by the Windows API so
-the user must somehow transmit the size of the segment to processes opening the
-segment.
+shared memory.
 
 Sharing memory between services and user applications is also different. To share memory
 between services and user applications the name of the shared memory must start with the
@@ -865,7 +860,7 @@
 object or a file mapping are the same class and this has many advantages.
 
 One can, for example, mix in STL containers mapped regions from shared memory
-and memory mapped files. The libraries that only depend on mapped regions can
+and memory mapped files. Libraries that only depend on mapped regions can
 be used to work with shared memory or memory mapped files without recompiling them.
 
 [endsect]
@@ -881,7 +876,7 @@
 in a different way (allocation of more or less dynamic memory, for example), there is
 no guarantee that the file/shared memory is going to be mapped in the same address.
 
-If two processes map the same object in different addresses, this invalids the use
+If two processes map the same object in different addresses, this invalidates the use
 of pointers in that memory, since the pointer (which is an absolute address) would
 only make sense for the process that wrote it. The solution for this is to use offsets
 (distance) between objects instead of pointers: If two objects are placed in the same
@@ -6574,6 +6569,8 @@
 
 [section:notes_windows Notes for Windows users]
 
+[section:notes_windows_com_init COM Initialization]
+
 [*Boost.Interprocess] uses the COM library to implement some features and initializes
 the COM library with concurrency model `COINIT_APARTMENTTHREADED`.
 If the COM library was already initialized by the calling thread for other model, [*Boost.Interprocess]
@@ -6588,6 +6585,44 @@
 
 [endsect]
 
+[section:notes_windows_shm_folder Shared memory emulation folder]
+
+Shared memory (`shared_memory_object`) is implemented in windows using memory mapped files, placed in a
+directory in the shared documents folder (`SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders\Common AppData`).
+This directory name is the last bootup time (obtained via COM calls), so that each bootup shared memory is created in a new
+folder obtaining kernel persistence shared memory.
+
+Unfortunately, due to COM implementation related errors, in Boost 1.48 & Boost 1.49 the bootup-time folder was dumped and files
+were directly created in shared documents folder, reverting to filesystem persistence shared memory. Boost 1.50 fixed those issues
+and recovered bootup time directory and kernel persistence. If you need to reproduce Boost 1.48 & Boost 1.49 behaviour to communicate
+with applications compiled with that version, comment `#define BOOST_INTERPROCESS_HAS_KERNEL_BOOTTIME` directive
+in the Windows configuration part of `boost/interprocess/detail/workaround.hpp`.
+
+[endsect]
+
+[endsect]
+
+[section:notes_linux Notes for Linux users]
+
+[section:notes_linux_overcommit Overcommit]
+
+The committed address space is the total amount of virtual memory (swap or physical memory/RAM) that the kernel might have to supply
+if all applications decide to access all of the memory they've requested from the kernel.
+By default, Linux allows processes to commmit more virtual memory than available in the system. If that memory is not
+accessed, no physical memory + swap is actually used.
+
+The reason for this behaviour is that Linux tries to optimize memory usage on forked processes; fork() creates a full copy of
+the process space, but with overcommitted memory, in this new forked instance only pages which have been written to actually need
+to be allocated by the kernel. If applications access more memory than available, then the kernel must free memory in the hard way:
+the OOM (Out Of Memory)-killer picks some processes to kill in order to recover memory.
+
+[*Boost.Interprocess] has no way to change this behaviour and users might suffer the OOM-killer when accessing shared memory.
+According to the [@http://www.kernel.org/doc/Documentation/vm/overcommit-accounting Kernel documentation], the
+Linux kernel supports several overcommit modes. If you need non-kill guarantees in your application, you should
+change this overcommit behaviour.
+
+[endsect]
+
 [endsect]
 
 [section:thanks_to Thanks to...]
@@ -6635,6 +6670,19 @@
 
 [section:release_notes Release Notes]
 
+[section:release_notes_boost_1_52_00 Boost 1.52 Release]
+
+*  Added `shrink_by` and `advise` functions in `mapped_region`.
+*  Reimplemented `message_queue` with a circular buffer index (the
+   old behavior used an ordered array, leading to excessive copies). This
+   should greatly increase performance but breaks ABI. Old behaviour/ABI can be used
+   undefining macro `BOOST_INTERPROCESS_MSG_QUEUE_CIRCULAR_INDEX` in `boost/interprocess/detail/workaround.hpp`
+*  Improved `message_queue` insertion time avoiding priority search for common cases
+   (both array and circular buffer configurations).
+
+[endsect]
+
+
 [section:release_notes_boost_1_51_00 Boost 1.51 Release]
 
 *  Synchronous and asynchronous flushing for `mapped_region::flush`.

Modified: trunk/libs/interprocess/proj/vc7ide/message_queue.vcproj
==============================================================================
--- trunk/libs/interprocess/proj/vc7ide/message_queue.vcproj	(original)
+++ trunk/libs/interprocess/proj/vc7ide/message_queue.vcproj	2012-08-07 05:13:23 EDT (Tue, 07 Aug 2012)
@@ -13,7 +13,7 @@
                 <Configuration
                         Name="Debug|Win32"
                         OutputDirectory="../../Bin/Win32/Debug"
-			IntermediateDirectory="Debug/shared_message_queue_test"
+			IntermediateDirectory="Debug/message_queue_test"
                         ConfigurationType="1"
                         CharacterSet="2">
                         <Tool
@@ -35,7 +35,7 @@
                         <Tool
                                 Name="VCLinkerTool"
                                 AdditionalDependencies="winmm.lib"
-				OutputFile="$(OutDir)/shared_message_queue_test_d.exe"
+				OutputFile="$(OutDir)/message_queue_test_d.exe"
                                 LinkIncremental="1"
                                 AdditionalLibraryDirectories="../../../../stage/lib"
                                 GenerateDebugInformation="TRUE"
@@ -67,7 +67,7 @@
                 <Configuration
                         Name="Release|Win32"
                         OutputDirectory="../../Bin/Win32/Release"
-			IntermediateDirectory="Release/shared_message_queue_test"
+			IntermediateDirectory="Release/message_queue_test"
                         ConfigurationType="1"
                         CharacterSet="2">
                         <Tool
@@ -86,7 +86,7 @@
                         <Tool
                                 Name="VCLinkerTool"
                                 AdditionalDependencies="winmm.lib"
-				OutputFile="$(OutDir)/shared_message_queue_test.exe"
+				OutputFile="$(OutDir)/message_queue_test.exe"
                                 LinkIncremental="1"
                                 AdditionalLibraryDirectories="../../../../stage/lib"
                                 GenerateDebugInformation="TRUE"

Modified: trunk/libs/interprocess/test/message_queue_test.cpp
==============================================================================
--- trunk/libs/interprocess/test/message_queue_test.cpp	(original)
+++ trunk/libs/interprocess/test/message_queue_test.cpp	2012-08-07 05:13:23 EDT (Tue, 07 Aug 2012)
@@ -48,6 +48,8 @@
       message_queue::size_type recvd = 0;
       unsigned int priority = 0;
       std::size_t tstamp;
+      unsigned int priority_prev;
+      std::size_t  tstamp_prev;
 
       //We will send 100 message with priority 0-9
       //The message will contain the timestamp of the message
@@ -56,8 +58,31 @@
          mq1.send(&tstamp, sizeof(tstamp), (unsigned int)(i%10));
       }
 
-      unsigned int priority_prev = (std::numeric_limits<unsigned int>::max)();
-      std::size_t  tstamp_prev = 0;
+      priority_prev = (std::numeric_limits<unsigned int>::max)();
+      tstamp_prev = 0;
+
+      //Receive all messages and test those are ordered
+      //by priority and by FIFO in the same priority
+      for(std::size_t i = 0; i < 100; ++i){
+         mq1.receive(&tstamp, sizeof(tstamp), recvd, priority);
+         if(priority > priority_prev)
+            return false;
+         if(priority == priority_prev &&
+            tstamp   <= tstamp_prev){
+            return false;
+         }
+         priority_prev  = priority;
+         tstamp_prev    = tstamp;
+      }
+
+      //Now retry it with different priority order
+      for(std::size_t i = 0; i < 100; ++i){
+         tstamp = i;
+         mq1.send(&tstamp, sizeof(tstamp), (unsigned int)(9 - i%10));
+      }
+
+      priority_prev = (std::numeric_limits<unsigned int>::max)();
+      tstamp_prev = 0;
 
       //Receive all messages and test those are ordered
       //by priority and by FIFO in the same priority
@@ -83,7 +108,7 @@
 //another buffer and checks it against the original data-base
 bool test_serialize_db()
 {
-   //Typedef data to create a Interprocess map  
+   //Typedef data to create a Interprocess map
    typedef std::pair<const std::size_t, std::size_t> MyPair;
    typedef std::less<std::size_t>   MyLess;
    typedef node_allocator<MyPair, managed_external_buffer::segment_manager>
@@ -157,7 +182,7 @@
             break;
          }
       }
-     
+
       //The buffer will contain a copy of the original database
       //so let's interpret the buffer with managed_external_buffer
       managed_external_buffer db_destiny(open_only, &buffer_destiny[0], BufferSize);
@@ -188,7 +213,7 @@
             return false;
          }
       }
-     
+
       //Destroy maps from db-s
       db_origin.destroy_ptr(map1);
       db_destiny.destroy_ptr(map2);