[yt-users] yt3.3dev MPI puzzle

[Stuart Levy]

Hello yt people,

I'm trying to use MPI on a simple Enzo volume-rendering script, using
the latest yt33 dev from bitbucket, and am getting a puzzling
behavior.   (MPI so as to reduce the load time.)

Symptom: the datatype of the Scene.last_render field is inconsistent:
it's YTImageArray when successful, but just plain YTArray when using MPI
on the right (wrong) data, and so sc.save() fails.    It's easy to work
around and save the image another way, but makes me worried that
something else is wrong, or that I'm misusing yt.

It is somehow data-dependent.   The error doesn't happen when I use the
tiny enzo dataset in .../yt-hg/tests/DD0010/moving7_0010, nor the
enzo_tiny_cosmology sample (DD0043).   But it *does* happen when I use a
somewhat-larger-but-not-huge simulation from John Wise - the
low-resolution PopIII-star simulation he'd given us last year.   A
single timestep from it is ~46MB and has only a few dozen grids.

I could post the data somewhere if John Wise would be OK with that - John?

Demo code in:
    http://paste.yt-project.org/show/6343/
And its output when run under mpirun:
    http://paste.yt-project.org/show/6344/

The one change to the yt code from 'tip' (bed01b2c838c) is this diagnostic:
diff -r bed01b2c838c yt/visualization/volume_rendering/scene.py
--- a/yt/visualization/volume_rendering/scene.py    Mon Mar 21 14:15:54
2016 -0700
+++ b/yt/visualization/volume_rendering/scene.py    Mon Mar 21 18:01:20
2016 -0500
@@ -284,6 +284,7 @@
             self.render()
 
         mylog.info("Saving render %s", fname)
+    print "about to write_png() - Scene.last_render",
type(self.last_render), "shape",self.last_render.shape, " => ",
dir(self.last_render)
         self.last_render.write_png(fname, sigma_clip=sigma_clip)