[yt-dev] Fwd: [mpi4py] Fwd: [Numpy-discussion] Improving Python+MPI import performance

[Matthew Turk]

Hi Britton,

My fix has no sideeffects. I think it is safe to apply.

I spoke to Lisandro Dalcin from mpi4py today and he said it is possible
that this will be included in mpi4py proper at some point, so once that
happens we should deprecated ours.

I am of the opinion we can do this with an alternate, parallel import that
would be compatible with yt.mods. Something like yt.pmods. What do you
think? What should usage of this be?

Matt
On Jan 16, 2012 10:48 AM, "Britton Smith" <brittonsmith at gmail.com> wrote:

> Matt,
>
> Does the fix you posted for this have any negative side-effects that would
> prevent us from officially adopting this as a solution?  If not, should we
> try to integrate this into yt or update some documentation so people know
> that they should be using this?
>
> Britton
>
> On Sun, Jan 15, 2012 at 1:01 PM, Stephen Skory <s at skory.us> wrote:
>
>> Hi Britton & Matt,
>>
>> > I just tested this new import method with Matt's fix on Kraken against
>> the
>> > standard import for a variety of core counts.  In the attached figure, I
>> > plot the mean time to import, with errorbars showing the minimum and
>> maximum
>> > time for all cores.  I think the plot speaks for itself.
>>
>> I just tried on Janus, and while the new import is faster, it's not
>> quite as impressive as what Britton sees, it's definitely not slower.
>> At 1024 threads it went from 18 sec to 16. But clearly that disk is
>> much nicer to work with than Kraken's lustre. So, I figure we should
>> use this!
>>
>> --
>> Stephen Skory
>> s at skory.us
>> http://stephenskory.com/
>> 510.621.3687 (google voice)
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20120116/cb1af731/attachment.html>