[yt-users] Running yt in parallel on a subset of available processes

Matthew Turk matthewturk at gmail.com
Thu Dec 4 07:05:43 PST 2014


Hi Dan,

Yup, you can commit in that directory, and push to a fork.  Create a
fork on bitbucket, then do your changes and commit, and issue this
command:

hg push https://bitbucket.org/yourusername/yt

Then you can issue a pull request via their web interface.  You can
also edit your hgrc to add an alias, but that can happen later.

-Matt

On Thu, Dec 4, 2014 at 8:08 AM, Daniel Fenn <dsfenn at gmail.com> wrote:
> Hi Cameron,
>
> Yeah, no problem. I'm not exactly sure how to do that. Do I have to fork yt
> first, or can I use my existing installation since it's already using
> Mercurial? I'm familiar with subversion, since that's what we use in our
> research group, but I'm a bit hazy on how the whole thing is set up with yt.
> When I installed yt using the installation script, it created a yt-hg
> directory. Is that a checked-out version of the code that I can commit?
>
> Thanks!
>
> Dan
>
> 2014-12-03 20:59 GMT-05:00 Cameron Hummels <chummels at gmail.com>:
>
>> Hey Daniel,
>>
>> This is great that you got this working, and it sounds like something from
>> which others could benefit! Would you mind submitting this change to the
>> main codebase in a pull request?
>>
>> Cameron
>>
>> On Wed, Dec 3, 2014 at 6:46 PM, Daniel Fenn <dsfenn at gmail.com> wrote:
>>>
>>> Hi Matt,
>>>
>>> I was able to get my MPI problem fixed by using subcommunicators. I
>>> modified enable_parallelism() to take an optional argument that specifies
>>> the MPI communicator yt should use. If none is specified, it defaults to
>>> COMM_WORLD. Seems to be working pretty well so far.
>>>
>>> Thanks for the advice,
>>>
>>> Dan
>>>
>>> 2014-12-03 19:16 GMT-05:00 Matthew Turk <matthewturk at gmail.com>:
>>>
>>>> Hi Dan,
>>>>
>>>> I've done this before by modifying how the global communicator is
>>>> created.  The way they work in yt is through pushing new communicators
>>>> onto a stack, which can vary based on which processors you're on.  I
>>>> don't think I ever wrote it up in a PR, but I think it could be done
>>>> without too much work.
>>>>
>>>> On Wed, Dec 3, 2014 at 11:47 AM, Daniel Fenn <dsfenn at gmail.com> wrote:
>>>> > Is there a way to restrict yt to only use a subset of the available
>>>> > processes when it's running in parallel? I have a set of ipython
>>>> > engines
>>>> > running, and I'm trying to launch a yt script on a subset of them, but
>>>> > it
>>>> > just hangs. It works fine if I launch the same script on all the
>>>> > processes.
>>>> >
>>>> > Printing out MPI.COMM_WORLD.Get_size() from the script gives the total
>>>> > number of engines, not the size of the subset I'm trying to use.
>>>> >
>>>> > I haven't done a lot of parallel programming, but I would guess that
>>>> > the
>>>> > running processes are waiting on one that's not running a yt script,
>>>> > since
>>>> > they can see the total number of processes available, but not the size
>>>> > of
>>>> > the subset of processes that are actually running a yt script.
>>>> >
>>>> > Does anyone have any thoughts on how to do this?
>>>> >
>>>> > Thanks,
>>>> >
>>>> > Dan
>>>> >
>>>> > _______________________________________________
>>>> > yt-users mailing list
>>>> > yt-users at lists.spacepope.org
>>>> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>> >
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>
>>
>>
>> --
>> Cameron Hummels
>> Postdoctoral Researcher
>> Steward Observatory
>> University of Arizona
>> http://chummels.org
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>



More information about the yt-users mailing list