Re: [Boost-users] hybrid parallelism

Subject: Re: [Boost-users] hybrid parallelism
From: Brian Budge (brian.budge_at_[hidden])
Date: 2010-11-03 19:16:41

Hi Hicham -

Yes, you can use MPI (possibly through boost::mpi) to distribute tasks
to multiple machines, and then use threads on those machines to work
on finer grained portions of those tasks.  From another thread on this
list, there are constructs in boost::asio that handle task queuing for
the thread tasks.

  Brian

On Wed, Nov 3, 2010 at 3:43 PM, Hicham Mouline <hicham_at_[hidden]> wrote:
> Hello,
>
> Following a previous thread that asked about how to parallelize a large
> number of calculations, I took this summary:
>
>
>
> I am thinking of choosing a simple model whereby:
>
> There are M computers (possibly heterogeneous). I will stick to 1 process
> per computer.
>
> Each process will have N threads. Giving a total M*N "execution units".
>
> I will take a simple solution in that the M and N are fixed though
> determined at runtime.
>
>
>
> So there is this pool of M*N exec units and I give them 100 000 tasks to do.
>
> Those tasks get scheduled in the same process/computer, they share their
> memory.
>
> Each of the computers gets a duplicate of the memory used as inputs to the
> tasks.
>
>
>
> For thread pool, TBB, boost::asio were suggested. I believe there is also
> threadpool.sourceforge.net
>
> For the cross-computer communication, boost::mpi.
>
>
>
> Is this something boost::mpi + mpi can help with?
>
>
>
> Regards,
>
> _______________________________________________
> Boost-users mailing list
> Boost-users_at_[hidden]
> http://lists.boost.org/mailman/listinfo.cgi/boost-users
>