[yt-users] Running yt in parallel on a subset of available processes

[Cameron Hummels]

Hey Daniel,

This is great that you got this working, and it sounds like something from
which others could benefit! Would you mind submitting this change to the
main codebase in a pull request?

Cameron

On Wed, Dec 3, 2014 at 6:46 PM, Daniel Fenn <dsfenn at gmail.com> wrote:

> Hi Matt,
>
> I was able to get my MPI problem fixed by using subcommunicators. I
> modified enable_parallelism() to take an optional argument that specifies
> the MPI communicator yt should use. If none is specified, it defaults to
> COMM_WORLD. Seems to be working pretty well so far.
>
> Thanks for the advice,
>
> Dan
>
> 2014-12-03 19:16 GMT-05:00 Matthew Turk <matthewturk at gmail.com>:
>
> Hi Dan,
>>
>> I've done this before by modifying how the global communicator is
>> created.  The way they work in yt is through pushing new communicators
>> onto a stack, which can vary based on which processors you're on.  I
>> don't think I ever wrote it up in a PR, but I think it could be done
>> without too much work.
>>
>> On Wed, Dec 3, 2014 at 11:47 AM, Daniel Fenn <dsfenn at gmail.com> wrote:
>> > Is there a way to restrict yt to only use a subset of the available
>> > processes when it's running in parallel? I have a set of ipython engines
>> > running, and I'm trying to launch a yt script on a subset of them, but
>> it
>> > just hangs. It works fine if I launch the same script on all the
>> processes.
>> >
>> > Printing out MPI.COMM_WORLD.Get_size() from the script gives the total
>> > number of engines, not the size of the subset I'm trying to use.
>> >
>> > I haven't done a lot of parallel programming, but I would guess that the
>> > running processes are waiting on one that's not running a yt script,
>> since
>> > they can see the total number of processes available, but not the size
>> of
>> > the subset of processes that are actually running a yt script.
>> >
>> > Does anyone have any thoughts on how to do this?
>> >
>> > Thanks,
>> >
>> > Dan
>> >
>> > _______________________________________________
>> > yt-users mailing list
>> > yt-users at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>> >
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>


-- 
Cameron Hummels
Postdoctoral Researcher
Steward Observatory
University of Arizona
http://chummels.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20141203/3e3e5c64/attachment.html>