[interprocesss] managed_mapped_file mutex question
This is a somewhat different take on the question I asked earlier about interprocess mutexes in allocators. I am using a default interprocess::managed_mapped_file to hold a index for a multiprocess property store. There is one object named "index" in the mapped file, which is an interprocess::map. The map holds an interprocess::set of fixed-length strings. (I had an allocation problem with sets of string objects, but that's another topic) Many processes will both read and write this index map. Do the interprocess:: containers provide the needed synchronization, or do I need other synchronization mechanisms? If I am reading the docs correctly, it seems to say that I don't need additional synchronization if I am not adding/removing named objects. However, my tests break with multiple processes. I don't know if I just need my own synchronization or if a deeper problem is implied. Following is the simple test I have been using. Works with one process, breaks with two concurrent. (os-x 10.5, boost trunk) Thanks in advance for any advice! Andy ------------------------------------------------------- namespace bip = boost::interprocess; typedef struct { char t[64]; } thing_t; struct ThingLess { bool operator()(thing_t const & lhs, thing_t const & rhs) const { return (strcmp(lhs.t, rhs.t) < 0); } }; typedef bip::managed_mapped_file shared_store_t; typedef shared_store_t::segment_manager segment_manager_t; typedef bip::allocator<void, segment_manager_t> void_allocator_t; typedef bip::allocator<char, segment_manager_t> char_allocator_t; typedef bip::basic_string<char, std::char_traits<char>, char_allocator_t> char_string_t; typedef bip::allocator<char_string_t, segment_manager_t> char_string_allocator_t; typedef bip::allocator<thing_t, segment_manager_t> thing_allocator_t; typedef bip::set<thing_t, ThingLess, thing_allocator_t> thing_set_t; typedef std::pair<const char_string_t, thing_set_t> map_value_t; typedef bip::allocator<map_value_t, segment_manager_t> map_value_allocator_t; typedef bip::map< char_string_t, thing_set_t, std::less<char_string_t>, map_value_allocator_t> index_map_t; int main(int argc, char* argv[]) { //fork(); // forking here will usually cause a bus error int pid = getpid(); // find or create the mapped file std::auto_ptr<shared_store_t> store(new shared_store_t(bip::open_or_create, "/tmp/testindex", 1024*1024)); void_allocator_t allocator(store->get_segment_manager()); // find or create the index object index_map_t* index = store->get_segment_manager()-
find<index_map_t>("index").first; if (!index) { index = store->construct<index_map_t>("index") (std::less<char_string_t>(), allocator); assert(index); }
// add pairs to the map, find them, remove them. for (int j=0; j<1000; ++j) { for (int i=0; i<100; ++i) { try { std::ostringstream ks; ks << "key" << i; std::ostringstream ts; ts << "thing" << i; // look for thing in map char_string_t key(ks.str().c_str(), allocator); index_map_t::iterator foundIt = index->find(key); if (foundIt == index->end()) { // add thing to map thing_t thing; strcpy(thing.t,ts.str().c_str()); thing_set_t thingSet(ThingLess(), allocator); thingSet.insert(thing); map_value_t entry(key,thingSet); index->insert(entry); std::cout << pid << " Added " << key << std::endl; } // look again... foundIt = index->find(key); if (foundIt != index->end()) { if (ts.str() == foundIt->second.begin()->t) { std::cout << pid << " FOUND " << key << ":"<< foundIt-
second.begin()->t << std::endl; } else { std::cout << pid << " FAILED? " << key << ":"<< foundIt- second.begin()->t << std::endl; } } if (j%2) { // erase thing from map, every other pass just for some variety index->erase(key); std::cout << pid << " Erased " << key << std::endl; } } catch( bip::bad_alloc & ba ) { std::cout << "bad_alloc at " << j<<","<<i<< " "<< ba.what() << std::endl; } catch(std::exception & e) { std::cout << e.what() << std::endl; } catch(...) { std::cout << "some other exception" << std::endl; } } std::cout << pid << " --------- End Round " << j << " --------- " <<std::endl; }
return 0; }
El 18/03/2010 16:55, Andy Wiese escribió:
Many processes will both read and write this index map. Do the interprocess:: containers provide the needed synchronization, or do I need other synchronization mechanisms?
You need to provide them. Interprocess memory allocation (via construct<>, or allocate(), or through allocators) is thread-safe, just like ::new is thread safe for inter-thread programming. But similarly to inter-thread programming interprocess::map offers the same guarantees as std::map, because interprocess containers are standard-conforming containers: you must provide your own synchronization if you use the same object by two processes-theads and one is inserting or erasing elements. Best, Ion
On Mar 18, 2010, at 8:43 PM, Ion Gaztañaga wrote:
El 18/03/2010 16:55, Andy Wiese escribió:
Many processes will both read and write this index map. Do the interprocess:: containers provide the needed synchronization, or do I need other synchronization mechanisms?
You need to provide them. Interprocess memory allocation (via construct<>, or allocate(), or through allocators) is thread-safe, just like ::new is thread safe for inter-thread programming. But similarly to inter-thread programming interprocess::map offers the same guarantees as std::map, because interprocess containers are standard-conforming containers: you must provide your own synchronization if you use the same object by two processes-theads and one is inserting or erasing elements.
Best,
Ion
Thank you. That makes perfect sense and is now proven by my test program. I have a couple of questions about crash protection, which is very important in this case: 1. I am using only file_lock, because it is guaranteed to be released if the process crashes (if I understand correctly). However, file_lock has its limitations. If I use a named_recursive_mutex instead, is there any way to clear the mutex if the process terminates while holding the mutex? 2. If the mapped file gets corrupted, e.g. if a process gets killed while writing the file, I need to be able to detect that case and rebuild the file, but the file is certain death to any process that touches it, so validating it seems messy. Any suggestions how to detect a bad file? Andy
El 19/03/2010 6:00, Andy Wiese escribió:
I have a couple of questions about crash protection, which is very important in this case: 1. I am using only file_lock, because it is guaranteed to be released if the process crashes (if I understand correctly). However, file_lock has its limitations. If I use a named_recursive_mutex instead, is there any way to clear the mutex if the process terminates while holding the mutex?
No, this is a difficult problem to solve. I plan to work on this when I get some time but this is one of the weak points of the library.
2. If the mapped file gets corrupted, e.g. if a process gets killed while writing the file, I need to be able to detect that case and rebuild the file, but the file is certain death to any process that touches it, so validating it seems messy. Any suggestions how to detect a bad file?
This depends on your application, and it's similar to when a thread corrupts a data structure used by another thread. You could serialize them and mark the start and end of a modification, so that a thread could check if a previous process/thread has finished the modification. This obviously, requires collaborative processes. Best, Ion
Andy
On 3/19/2010 6:06 AM, Ion Gaztañaga wrote:
El 19/03/2010 6:00, Andy Wiese escribió:
I have a couple of questions about crash protection, which is very important in this case: 1. I am using only file_lock, because it is guaranteed to be released if the process crashes (if I understand correctly). However, file_lock has its limitations. If I use a named_recursive_mutex instead, is there any way to clear the mutex if the process terminates while holding the mutex?
No, this is a difficult problem to solve. I plan to work on this when I get some time but this is one of the weak points of the library.
Ion, Thanks for all your hard work on Interprocess, it's a great library. Do you mind giving us some ideas about how you plan to solve that problem? I'm in a position where I need to solve it, maybe I can contribute, or at least use ideas you may have as a base to solve it on my own. One of my main issues is that the library seems to be designed around a "single parent, multiple children" style. In my situation, FastCGI (and the web server) launches my processes, and the children cannot communicate with their parent. They're all independent. The problem I'm having is that without a parent process to control what happens to the children, they become lost and cannot function if one of them crashes.
El 19/03/2010 19:16, Brett Gmoser escribió:
Thanks for all your hard work on Interprocess, it's a great library. Do you mind giving us some ideas about how you plan to solve that problem? I'm in a position where I need to solve it, maybe I can contribute, or at least use ideas you may have as a base to solve it on my own.
I would like to think about emulating "robust mutexes" somehow. I don't know how, and I suspect it will be really hard. But at least I would like to detect if a process has died holding a mutex when another process tries to lock it. That would impose also some overhead (each time you lock a mutex you need to check if a process has died... Not an easy task, Ion
I have a couple of questions about crash protection, which is very important in this case: 1. I am using only file_lock, because it is guaranteed to be released if the process crashes (if I understand correctly). However, file_lock has its limitations. If I use a named_recursive_mutex instead, is there any way to clear the mutex if the process terminates while holding the mutex? 2. If the mapped file gets corrupted, e.g. if a process gets killed while writing the file, I need to be able to detect that case and rebuild the file, but the file is certain death to any process that touches it, so validating it seems messy. Any suggestions how to detect a bad file?
Andy
Yeah, I think our problems are pretty similar. Unfortunately I haven't gotten much input on it - I posted it before, and didn't get and responses. I posted it again because I had seen that Ion was active here recently. The only input I can offer you (I'm not much of an expert with the Interprocess library) is that I believe that named_recursive_mutex also has filesystem persistence. It creates files in /tmp/boost_interprocess/XXXXX/name, where XXXXX seems to be a hash of the machine's boot time.
On Mar 19, 2010, at 11:31 AM, Brett Gmoser wrote:
I have a couple of questions about crash protection, which is very important in this case: 1. I am using only file_lock, because it is guaranteed to be released if the process crashes (if I understand correctly). However, file_lock has its limitations. If I use a named_recursive_mutex instead, is there any way to clear the mutex if the process terminates while holding the mutex? 2. If the mapped file gets corrupted, e.g. if a process gets killed while writing the file, I need to be able to detect that case and rebuild the file, but the file is certain death to any process that touches it, so validating it seems messy. Any suggestions how to detect a bad file?
Andy
Yeah, I think our problems are pretty similar. Unfortunately I haven't gotten much input on it - I posted it before, and didn't get and responses. I posted it again because I had seen that Ion was active here recently.
The only input I can offer you (I'm not much of an expert with the Interprocess library) is that I believe that named_recursive_mutex also has filesystem persistence. It creates files in /tmp/ boost_interprocess/XXXXX/name, where XXXXX seems to be a hash of the machine's boot time.
Referencing Ion's last reply. I think I will stick with file_lock for now, which I am able to wrap my own classes around to make an adequate recursive mutex. I have been using the flock style utilities on os-x and freebsd for some time now, and it seems to be dependable. I find myself wondering what are the conditions in which I would ever be comfortable using another lock type. I suppose if I had strict control and confidence in all processes, which is certainly not my current case. As for data corruption, I was also thinking along the lines of Ion's suggestion:
This depends on your application, and it's similar to when a thread corrupts a data structure used by another thread. You could serialize them and mark the start and end of a modification, so that a thread could check if a previous process/thread has finished the modification. This obviously, requires collaborative processes.
Accepting the constraint that I am trying to protect only against a file writer leaving a corrupt file by terminating abruptly, and given that file writers always have exclusive serialized access, based upon a file-based file_lock--the writer will write a token into the lock file upon start of write, and then clear that token upon completion. If either a reader or writer acquires the file_lock and finds that there is a token already written into the lock file, I assume that a writer failed to complete and the mapped file is suspect. Seems like a simple and obvious pattern, which I'm surprised I've never used before. I'm happy for suggestions if I've missed something. Brett it sounds like our applications are quite similar. Feel free to email me directly if you want to kibitz. Andy
participants (3)
-
Andy Wiese -
Brett Gmoser -
Ion Gaztañaga