[Yt-dev] Parallelism

[Cameron Hummels]

+1, +1

On 08/03/2011 12:08 PM, Britton Smith wrote:
> Going along with what Matt said, I think one place we can do a lot of
> consolidation is type specific communication functions.  For example,
> one function that does array concatenation and takes a type keyword
> arg is better than a separate function for each type.
>
> In addition, I think the goal of this exercise, which is similar to
> the original point of making a parallel analysis interface, is to
> provide a toolkit that allows people to develop parallel analysis
> without having to worry too much about the actual communication.  In
> the beginning, we really only had in mind parallel operations that
> were aware of the total number of processors and their own rank.  And
> all of this was operating within the scope of working on a single pf. 
> We have since then moved into grander things like performing analysis
> on multiple datasets to create a single analysis product.  The
> original PAI succeeded for a long time, but when we started to outgrow
> its scope, the only solution was to bolt new functions onto the side
> to accomplish what we needed.  So I think Matt's general point is that
> it is time for us to think about building something new that
> encompasses all the things we want to be able to do and hopefully
> more.  This is something I am definitely in favor of.
>
> Britton
>
> On Wed, Aug 3, 2011 at 8:41 AM, Stephen Skory <s at skory.us
> <mailto:s at skory.us>> wrote:
>
>     Matt,
>
>     > parallel_concatenate
>     > parallel_operation (max, min, etc)
>     > parallel_broadcast
>     >
>     > Would this be a good compromise?
>
>     Yes, I think is is a good way to go. It keeps the names clearer
>     without being overly specific.
>
>
>     >> I'v already done some of this, and I'd be willing to help out
>     some more.
>     >
>     > Excellent!  I'd love it if these could show up in a .py file
>     under tests/ .
>
>     Didn't I send you that stuff a while ago? Or is that not what you're
>     thinking of?
>
>     --
>     Stephen Skory
>     s at skory.us <mailto:s at skory.us>
>     http://stephenskory.com/
>     510.621.3687 <tel:510.621.3687> (google voice)
>     _______________________________________________
>     Yt-dev mailing list
>     Yt-dev at lists.spacepope.org <mailto:Yt-dev at lists.spacepope.org>
>     http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
>
> _______________________________________________
> Yt-dev mailing list
> Yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-- 
Cameron Hummels
PhD Candidate, Astronomy Department of Columbia University
Public Outreach Director, Astronomy Department of Columbia University
NASA IYA New York State Student Ambassador
http://outreach.astro.columbia.edu 
PGP: 0x06F886E3

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20110803/dde41831/attachment.htm>