[yt-dev] Summary of a yt sprint

[Matthew Turk]

Hi all,

This last weekend, John, Nathan, Britton, Sam and I were fortunate
enough to be in the same place for a couple days and we had a sprint
on some yt functionality.  Almost all of the changes have gone into
the code base already, but we wanted to share them with you all.  Feel
free to chime in with questions, comments, or more information.

= Parallelism Improvements =

Britton implemented a pure-MPI task queuing system that works with the
existing parallel_objects machinery.  It is currently not threaded
(and opinions vary as to whether that would be productive or not) and
so one MPI task is dedicated as the controller node that distributes
objects.  This can either be manually used or you can simply supply
"dynamic=True" to the parallel_objects machinery.  I have to say, the
changes to yt's parallelism in the last year or so have been pretty
much my favorite improvements, and Britton really knocked this one out
of the park, in my opinion.  As I noted in my other email, with the
pull request for Reason, Britton also added an MPI-parallel backend
for Reason.  I won't repeat my email here, but if you want to test it
out you'll need Pyro4 and the launcher script in scripts/ .  This will
be documented in more detail when I finish up the remaining task,
which is to deliver intermediate payloads from the MPI controller back
to the web browser.

= Plot Window =

Perhaps Nathan would be better at describing this, but he's now
implemented a mechanism (built on Jeff's initial design) for
generating nice plots easily and complex plots straightforwardly.
This was discussed here:

http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/2012-June/002111.html

Nathan, anything you wanted to add?

= Reason =

This was described in my other email as well:

http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/2012-June/002117.html

and this PR:

https://bitbucket.org/yt_analysis/yt/pull-request/179/reason-gui-refactor-extjs4-server

If anybody can test this out, I'd appreciate it.  I did note this
weekend that when running simulations, I spied a few of the other
sprinters using it.  I think seeing developers using something means
it has "arrived."  (And I know Sam has a volume rendering widget in
mind ...)

= Ghost Zones =

John spent a good amount of time implementing a fast sibling finder,
as well as a ghost zone filling mechanism.  This is designed to
replace the existing mechanism of a smoothed covering grid, which
requires interpolation from the coarsest level up to the requested
level.  This should lead to a considerable performance increase when
doing anything having ghost zones.  I think this is going to be PR'd
sometime in the next while.

--

Anyway, I think I speak for everybody when I say it was a pretty
successful sprint, and we'd love to hear your feedback!

-Matt