[Yt-dev] QuadTree projection now works in parallel

[Matthew Turk]

John, Sam, Stephen,

Thanks very much for testing.  I think maybe this is just about ready
for production; sounds like we should push ahead with swapping it out
for 2.2.  The only remaining test is the light cone stuff, but I was
able to test the source-selection and that worked for me.  It would be
a real feather in our caps I think to release 2.2 that included a
completely new web GUI (which includes PyLab support as well as google
maps-style widgets!), a projection speedup of 3.5x, and a new field
system.  And, even a new logo!

There are a few more optimizations I believe I can apply, which I will
attempt to do over the next little while -- but they are less
invasive.  Mainly these involve moving from pre-generated arrays of
positions to generating positions inside the Cython code on an
as-needed basis.

John -- 350 s to 104 s is pretty good, I think.  We may now be IO
dominated, but if you supply the argument preload="all" it might cut
down even further on the run time.

Thanks, everyone.

-Matt

On Thu, Jun 2, 2011 at 1:24 PM, John Wise <jwise at astro.princeton.edu> wrote:
> Nice work, Matt!
>
>> pf.h.proj = pf.h.quad_proj
>
> Clever trick.  I didn't think about doing this before when running QuadProj in serial before.
>
> I just tested it on pleiades on 128 processes on a large nested grid simulation.  The run time decreased from 348s to 104s.  Very nice.  There were no errors, but there were some lingering RECV/SEND debugging messages.  Here are the stats on the dataset.
>
> level   # grids     # cells
> ---------------------------
>  0        512    134217728
>  1        850     25412184
>  2       1229    140608000
>  3      13775    283353768
>  4      41136    195020184
>  5      29687     74512760
>  6      14847     17540640
>  7       4104      3998824
>  8        527      1099896
>  9         10         6984
> ----------------------------
>        106677    875770968
>
> John
> _______________________________________________
> Yt-dev mailing list
> Yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>