<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Thank you, Sam! I think this makes sense. Except, in case (1),
do I need to do something to bring the AMRKDTree into the picture?
Or are you telling me that it is automatically constructed whenever
you load_uniform_grid(), or volume-render it?<br>
<br>
I think the available nodes have 64GB, so to load the whole ~600GB
might take at least 32 nodes or 1024 cores.<br>
<br>
Will let you know how it goes!<br>
<br>
<div class="moz-cite-prefix">On 11/7/14 11:08 AM, Sam Skillman
wrote:<br>
</div>
<blockquote
cite="mid:CAB4=1s0QpgRdGxf_3r4_ay3kb0VUKx0+ntBk+6zn=MJf4f+Ukw@mail.gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
Ack, my calculation of 256-512 cores is probably low... feel free
to push up much higher.<br>
<br>
<div class="gmail_quote">On Fri Nov 07 2014 at 9:03:51 AM Sam
Skillman <<a moz-do-not-send="true"
href="mailto:samskillman@gmail.com">samskillman@gmail.com</a>>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Stuart,
<div><br>
</div>
<div>
<div class="gmail_quote">On Thu Nov 06 2014 at 8:36:28 AM
Stuart Levy <<a moz-do-not-send="true"
href="mailto:salevy@illinois.edu" target="_blank">salevy@illinois.edu</a>>
wrote:<br>
</div>
</div>
<div>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Hello
all,<br>
<br>
We're hoping to use yt parallel volume rendering on a
very large generic<br>
brick - it's a simple rectangular unigrid slab, but
containing something<br>
like 1.5e11 points, so much too large for
load_uniform_grid() to load<br>
into memory in a single machine.<br>
</blockquote>
<div><br>
</div>
</div>
</div>
<div>
<div class="gmail_quote">
<div>Are you loading directly using something like
numpy.fromfile? If so, I think the easiest method would
be to replace that with a np.memmap (<a
moz-do-not-send="true"
href="http://docs.scipy.org/doc/numpy/reference/generated/numpy.memmap.html"
target="_blank">http://docs.scipy.org/doc/numpy/reference/generated/numpy.memmap.html</a>).
Once that is loaded, you should be able to use
load_uniform_grid.</div>
<div><br>
</div>
<div>At that point, there are two possible routes that
both may or may not work well. </div>
<div><br>
</div>
<div>1) Just try rendering with ~256-512 cores, and the
AMRKDTree should try to geometrically split the grid
before performing and I/O. </div>
<div>or</div>
<div>2) Use load_uniform_grid with the keyword nprocs=N (
for this size simulation, you probably need something
like 256-1024 processors depending on the memory per
core). This should do the equivalent thing to (1), but
it may hit the I/O here instead of in the kd-tree.</div>
<div><br>
</div>
<div>I *think* (1) should be your best option, but I
haven't tried rendering this large of a single-grid
output.</div>
<div><br>
</div>
<div>When you build the camera option, definitely start
out using the keyword "no_ghost=True", as this will
extrapolate rather than interpolate from boundary grids
to the vertices. The rendering quality won't be quite as
good but for unigrid simulations there isn't a
tremendous difference. </div>
<div><br>
</div>
<div>Let us know how that goes! I'd be very excited to
see images from such a large sim...</div>
</div>
</div>
<div>
<div class="gmail_quote">
<div><br>
</div>
<div>Sam </div>
</div>
</div>
<div>
<div class="gmail_quote">
<div> <br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
I imagine it wouldn't be hard to do the domain
decomposition by hand,<br>
loading a different chunk of grid into each MPI
process. But then<br>
what? What would it take to invoke the volume renderer
on each piece<br>
and composite them together? Would it help if the
chunks were stored<br>
in a KDTree? Is there some example (one of the
existing data loaders?)<br>
which I could follow?<br>
_______________________________________________<br>
yt-users mailing list<br>
<a moz-do-not-send="true"
href="mailto:yt-users@lists.spacepope.org"
target="_blank">yt-users@lists.spacepope.org</a><br>
<a moz-do-not-send="true"
href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org"
target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
</blockquote>
</div>
</div>
</blockquote>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
yt-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a>
<a class="moz-txt-link-freetext" href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a>
</pre>
</blockquote>
<br>
</body>
</html>