[Yt-dev] Parallelism

Britton Smith brittonsmith at gmail.com
Wed Aug 3 09:08:53 PDT 2011


Going along with what Matt said, I think one place we can do a lot of
consolidation is type specific communication functions.  For example, one
function that does array concatenation and takes a type keyword arg is
better than a separate function for each type.

In addition, I think the goal of this exercise, which is similar to the
original point of making a parallel analysis interface, is to provide a
toolkit that allows people to develop parallel analysis without having to
worry too much about the actual communication.  In the beginning, we really
only had in mind parallel operations that were aware of the total number of
processors and their own rank.  And all of this was operating within the
scope of working on a single pf.  We have since then moved into grander
things like performing analysis on multiple datasets to create a single
analysis product.  The original PAI succeeded for a long time, but when we
started to outgrow its scope, the only solution was to bolt new functions
onto the side to accomplish what we needed.  So I think Matt's general point
is that it is time for us to think about building something new that
encompasses all the things we want to be able to do and hopefully more.
This is something I am definitely in favor of.

Britton

On Wed, Aug 3, 2011 at 8:41 AM, Stephen Skory <s at skory.us> wrote:

> Matt,
>
> > parallel_concatenate
> > parallel_operation (max, min, etc)
> > parallel_broadcast
> >
> > Would this be a good compromise?
>
> Yes, I think is is a good way to go. It keeps the names clearer
> without being overly specific.
>
>
> >> I'v already done some of this, and I'd be willing to help out some more.
> >
> > Excellent!  I'd love it if these could show up in a .py file under tests/
> .
>
> Didn't I send you that stuff a while ago? Or is that not what you're
> thinking of?
>
> --
> Stephen Skory
> s at skory.us
> http://stephenskory.com/
> 510.621.3687 (google voice)
> _______________________________________________
> Yt-dev mailing list
> Yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20110803/17e85072/attachment.html>


More information about the yt-dev mailing list