[yt-users] Running yt in parallel on a subset of available processes

[Daniel Fenn]

Hi Cameron,

Yeah, no problem. I'm not exactly sure how to do that. Do I have to fork yt
first, or can I use my existing installation since it's already using
Mercurial? I'm familiar with subversion, since that's what we use in our
research group, but I'm a bit hazy on how the whole thing is set up with
yt. When I installed yt using the installation script, it created a yt-hg
directory. Is that a checked-out version of the code that I can commit?

Thanks!

Dan

2014-12-03 20:59 GMT-05:00 Cameron Hummels <chummels at gmail.com>:

> Hey Daniel,
>
> This is great that you got this working, and it sounds like something from
> which others could benefit! Would you mind submitting this change to the
> main codebase in a pull request?
>
> Cameron
>
> On Wed, Dec 3, 2014 at 6:46 PM, Daniel Fenn <dsfenn at gmail.com> wrote:
>
>> Hi Matt,
>>
>> I was able to get my MPI problem fixed by using subcommunicators. I
>> modified enable_parallelism() to take an optional argument that specifies
>> the MPI communicator yt should use. If none is specified, it defaults to
>> COMM_WORLD. Seems to be working pretty well so far.
>>
>> Thanks for the advice,
>>
>> Dan
>>
>> 2014-12-03 19:16 GMT-05:00 Matthew Turk <matthewturk at gmail.com>:
>>
>> Hi Dan,
>>>
>>> I've done this before by modifying how the global communicator is
>>> created.  The way they work in yt is through pushing new communicators
>>> onto a stack, which can vary based on which processors you're on.  I
>>> don't think I ever wrote it up in a PR, but I think it could be done
>>> without too much work.
>>>
>>> On Wed, Dec 3, 2014 at 11:47 AM, Daniel Fenn <dsfenn at gmail.com> wrote:
>>> > Is there a way to restrict yt to only use a subset of the available
>>> > processes when it's running in parallel? I have a set of ipython
>>> engines
>>> > running, and I'm trying to launch a yt script on a subset of them, but
>>> it
>>> > just hangs. It works fine if I launch the same script on all the
>>> processes.
>>> >
>>> > Printing out MPI.COMM_WORLD.Get_size() from the script gives the total
>>> > number of engines, not the size of the subset I'm trying to use.
>>> >
>>> > I haven't done a lot of parallel programming, but I would guess that
>>> the
>>> > running processes are waiting on one that's not running a yt script,
>>> since
>>> > they can see the total number of processes available, but not the size
>>> of
>>> > the subset of processes that are actually running a yt script.
>>> >
>>> > Does anyone have any thoughts on how to do this?
>>> >
>>> > Thanks,
>>> >
>>> > Dan
>>> >
>>> > _______________________________________________
>>> > yt-users mailing list
>>> > yt-users at lists.spacepope.org
>>> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>> >
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
>
> --
> Cameron Hummels
> Postdoctoral Researcher
> Steward Observatory
> University of Arizona
> http://chummels.org
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20141204/9e178049/attachment.htm>