[yt-users] Running yt in parallel on a subset of available processes

[Daniel Fenn]

Hi Matt,

I was able to get my MPI problem fixed by using subcommunicators. I
modified enable_parallelism() to take an optional argument that specifies
the MPI communicator yt should use. If none is specified, it defaults to
COMM_WORLD. Seems to be working pretty well so far.

Thanks for the advice,

Dan

2014-12-03 19:16 GMT-05:00 Matthew Turk <matthewturk at gmail.com>:

> Hi Dan,
>
> I've done this before by modifying how the global communicator is
> created.  The way they work in yt is through pushing new communicators
> onto a stack, which can vary based on which processors you're on.  I
> don't think I ever wrote it up in a PR, but I think it could be done
> without too much work.
>
> On Wed, Dec 3, 2014 at 11:47 AM, Daniel Fenn <dsfenn at gmail.com> wrote:
> > Is there a way to restrict yt to only use a subset of the available
> > processes when it's running in parallel? I have a set of ipython engines
> > running, and I'm trying to launch a yt script on a subset of them, but it
> > just hangs. It works fine if I launch the same script on all the
> processes.
> >
> > Printing out MPI.COMM_WORLD.Get_size() from the script gives the total
> > number of engines, not the size of the subset I'm trying to use.
> >
> > I haven't done a lot of parallel programming, but I would guess that the
> > running processes are waiting on one that's not running a yt script,
> since
> > they can see the total number of processes available, but not the size of
> > the subset of processes that are actually running a yt script.
> >
> > Does anyone have any thoughts on how to do this?
> >
> > Thanks,
> >
> > Dan
> >
> > _______________________________________________
> > yt-users mailing list
> > yt-users at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20141203/e9a1687f/attachment.html>