[yt-users] domain-decomposition for volume rendering huge brick?

[Sam Skillman]

Hi Stuart,

On Thu Nov 06 2014 at 8:36:28 AM Stuart Levy <salevy at illinois.edu> wrote:

> Hello all,
>
> We're hoping to use yt parallel volume rendering on a very large generic
> brick - it's a simple rectangular unigrid slab, but containing something
> like 1.5e11 points, so much too large for load_uniform_grid() to load
> into memory in a single machine.
>

Are you loading directly using something like numpy.fromfile?  If so, I
think the easiest method would be to replace that with a np.memmap (
http://docs.scipy.org/doc/numpy/reference/generated/numpy.memmap.html).
Once that is loaded, you should be able to use load_uniform_grid.

At that point, there are two possible routes that both may or may not work
well.

1) Just try rendering with ~256-512 cores, and the AMRKDTree should try to
geometrically split the grid before performing and I/O.
or
2) Use load_uniform_grid with the keyword nprocs=N ( for this size
simulation, you probably need something like 256-1024 processors depending
on the memory per core). This should do the equivalent thing to (1), but it
may hit the I/O here instead of in the kd-tree.

I *think* (1) should be your best option, but I haven't tried rendering
this large of a single-grid output.

When you build the camera option, definitely start out using the keyword
"no_ghost=True", as this will extrapolate rather than interpolate from
boundary grids to the vertices. The rendering quality won't be quite as
good but for unigrid simulations there isn't a tremendous difference.

Let us know how that goes!  I'd be very excited to see images from such a
large sim...

Sam



>
> I imagine it wouldn't be hard to do the domain decomposition by hand,
> loading a different chunk of grid into each MPI process.   But then
> what?   What would it take to invoke the volume renderer on each piece
> and composite them together?   Would it help if the chunks were stored
> in a KDTree?   Is there some example (one of the existing data loaders?)
> which I could follow?
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20141107/e6d6b65b/attachment.html>