Re: [Boost-users] boost C++ load data from file

Subject: Re: [Boost-users] boost C++ load data from file
From: Matthias Troyer (troyer_at_[hidden])
Date: 2010-07-14 00:54:58

On 12 Jul 2010, at 22:54, Jack Bryan wrote:

> Thanks for your code, 
> 
> I am working on a multi-computer Open MPI cluster system. 
> 
> If I put these data files in /home/mypath/folder, is it possible that all non-head nodes can access the files in the folder ? 
> 
> I need to load some data to some nodes, if all nodes can access the data, I do not need to load them to each node one by one. 
> 
> If multiple nodes access the same file to get data, is there conflict ? 
> 
> For example, 
> 
> fopen(myFile) by node 1, at the same time fopen(myFile) by node 2. 
> 
> Is it allowed to do that on MPI cluster without conflict ? 
> 
> Are there some boost MPI routines to support this ? 
> 
> I cannot find them in boost.org.
> 
> Any help is appreciated. 
> 
> Jack

Hi Jack,

You already got the answer in a previous e-mail:

On 12 Jul 2010, at 21:24, Binglong Xie wrote:
> If you use the C standard library function fopen, your file IO has nothing to do with MPI. You'd better check the file sharing/mapping support of the underlying OS of your MPI nodes. 

It all depends on the filesystem, but generally read-only access should be fine. However since the disks will probably be the slowest hardware part of your system it makes more sense that only one node reads each file and then broadcasts the contents to the other nodes that need it.

Matthias