[mpi] open/write to file for simple output

Hi, I am fairly new to mpi (and boost mpi). I have an mpi program and I need to dump some intermediate and results to a simple text file. First, is there already any good (using boost.mpi or not) to open and write files (let's say from process 0) and write to them. Second, if not, I am already trying to make the code as transparent as possible: I guess the most primitive way to do this is to do something like ofstream ofs(); if(world.rank()==0) ofs.open("file"); ... if(world.rank()==0) ofs<<result<<endl; but the code is full of "if(world.rank()==0)" and worst of all there is a dangling ofs. I tried other possibilities, like making intensive use of boost::optional boost::optional<std::ofstream> optional_ofs(world.rank()==0, std::ofstream("file") ); ... if(optional_ofs) *optional_ofs<<result<<endl; //or optional_ofs? *optional_ofs<<result<<endl; and even activating exceptions for boost optional, in such a way that this makes sense: try{ *optional_ofs<<result<<endl; }catch(...){} and executes only for the process(es) where the stream is actually created. The good thing is that once the stream is created it handles its own existence, regardless of the process number chosen to write to the file. I also looked into boost.interprocess file_lock but it seems it doesn't apply to this case. The third option is more complicated (although I partially implemented it already) by encapsulating this in a class such that the following syntax makes sense for the parallel execution mpi::ofstream ofs("file", world); // class mpi::ofstream is defined somehow, only one process actually opens the file ofs<<result<<endl; //(indirectly only the first process writes to the file This is even more general because (optionally) I can make the result to be printed only once if all processes agree in the results (the value of the variable is common) or different times (or just fail) if the actaul values of result do not agree from all the processes. does this all makes any sense? or I am reinventing the wheel? Also, in my particular cluster, all the processes share the same file system, which I think it is the common case but also the origin of the dilemma. (Note that I am *not* trying to write a file in parallel, I am just trying to print common (gathered) results to a file. nor I am trying to read from a file for the moment) Any advice? Thank you, Alfredo

On Mar 6, 2010, at 3:47 AM, alfC wrote:
I am fairly new to mpi (and boost mpi). I have an mpi program and I need to dump some intermediate and results to a simple text file.
Hi Alfredo, A common technique is to have each separate MPI process maintain it's own local buffer of data destined for the file, that is locally filled. When the buffer is full on the root processor, the buffer is written to the file. When the buffer is full on other processors, the size of the buffer is doubled so you don't lose any data. In time stepping applications it is common to have one place at the top or bottom of the time step where all processor's data is combined onto the root processor and then written to the file. This is a global synchronization point and every MPI process must participate even if it's local file buffers are empty (nothing written on that processor since the last time step). There are numerous possibilities for the local buffer ranging from a std::string all the way up to creating a specialized type from a std::streambuf. Hope that helps. -- Noel
participants (2)
-
alfC
-
Belcourt, Kenneth