[Yt-dev] Parallelism

[Stephen Skory]

Yo,

> Today at a meeting, it was mentioned that perhaps yt is having  trouble
> with parallelism.  To everyone out there: how reflective is this  of
> your experience?  Is yt okay with parallelism?   (Excluding
> projections, which I have a new engine ready to go  on.)


I think the biggest hurdle to parallelism is the cost of starting up the Python 
interpreter and loading the modules. It requires too many disk I/O operations to 
scale well. There is only one way to fix it, which is the various 
executable-gluing methods. Perhaps we should make a better effort to attempt to 
make this work on various machines, and document it?

As a follow-on, I've tried this various ways on Kraken, but due to the quirks of 
CNL, I was unsuccessful.

Besides this, I don't think it does. Did the comment have a context or 
qualification?

 _______________________________________________________
sskory at physics.ucsd.edu o__ Stephen Skory
http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student
________________________________(_)_\(_)_______________