<div dir="ltr">Hi Alex,<div><br></div><div>I think there are a couple things to clear up here -- hopefully this will help.</div><div><br></div><div>First a little background. For the case of patch-based AMR volume rendering, a popular technique is to first decompose the dataset into a kd-tree of single-resolution bricks that form the leaf nodes. This helps both ray traversal ordering and provides a way to tile the multi-resolution dataset into single-resolution bricks. When writing the AMRKDTree object in yt, we based the majority of the design off this paper: <a href="http://vis.lbl.gov/Publications/2002/LBNL-49954-AMRVolumeRendering.pdf">http://vis.lbl.gov/Publications/2002/LBNL-49954-AMRVolumeRendering.pdf</a><br>
</div><div><br></div><div>The technique is:</div><div>1. Take as input the AMR patches</div><div>2. Find splitting planes that produce as minimal set of bricks and to maximize load balance. This uses heuristics, and effects parallelization and i/o balance.</div>
<div>3. Provide an interface to iterate through these bricks in an arbitrary ordering defined by viewing positions.</div><div>4. For datasets larger than what can be held in memory, one needs to either stream in data from disk or decompose the volume in a way that can be later combined in a composite step.</div>
<div><br></div><div>A key thing to note here is that a single patch may be composed of many kd-tree bricks. This is why you see a mismatch between 'dims' of the brick and 'ActiveDimensions' of the grid (same goes for edge positions).</div>
<div><br></div><div>In yt, the steps are implemented as:</div><div><br></div><div>1. Use the hierarchy object as input</div><div>2. Look at the build() and add_grids function that hangs off the (internal API, maybe not well named) Tree object in yt/utilities/amr_kdtree/amr_kdtree.py</div>
<div>3. Look at depth_traverse, viewpoint_traverse, slice_traverse, etc.</div><div>4. There is a bunch of logic on how to decompose a kd-tree in parallel, and combine using reduce_tree_images. We domain decompose the tree in a breadth-first traversal over N processors.</div>
<div><br></div><div><br></div><div>Other notes:</div><div><br></div><div>For tri-linear interpolation, vertex centered data is generated through initialize_source and get_brick_data. I hope to work with folks to make the data access a bit simpler for different fields during the yt workshop in a few weeks.</div>
<div><br></div><div>For GPU volume rendering of AMR data, make sure to take a look at some of the literature out there, for example <a href="http://arxiv.org/abs/1212.3333">http://arxiv.org/abs/1212.3333</a></div><div><br>
</div><div>I am happy to help go through some of this at UCSC in a few weeks. I'm afraid I won't be able to dedicate a ton of time on this until then, however. The code in yt shouldn't be too difficult to parse though, if you want to really dive in you'll have to look in yt/utilities/lib/amr_kdtools.pyx for the Cython pieces. yt-3.0 has some more improvements -- I'm not sure what codebase you are working from.</div>
<div><br></div><div>Cheers,</div><div>Sam</div><div><br></div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Mar 10, 2014 at 1:20 PM, Alex Bogert <span dir="ltr"><<a href="mailto:bogart.alex@gmail.com" target="_blank">bogart.alex@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Moving our discussion about AMR hard ware volume rendering to yt-dev.:<div class="gmail_quote"><br>Hey Matt,<br>
<br>
The variables I need are the corners of the bounding box (which I call mi, ma, basically left and right edges), number of cells in a grid, grid data itself, children of a grid, and parent of a grid. This data itself works.<br>
<br>
I notice children aren't referenced. Is that because everything is defined just in terms of who the parent is, and where the tile exists within the parent? (Which if that's the case, since a top-down traversal is necessary in CUDA, the structure would have to be reversed).<br>
<br>
Another thing, I notice that 'dims' and g.ActiveDimensions do not match up a majority of the time. If they don't match up, how do you take a point in space and determine what grid element it maps to? (Trivial, but if these two don't match, I'm not sure who I should trust when doing the computation).<div>
<div><br>
<br>
<br>
<br>
On 3/10/14, 10:59 AM, Matthew Turk wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Alex,<br>
<br>
We can have these discussions on yt-dev, too:<br>
<br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/<u></u>listinfo.cgi/yt-dev-spacepope.<u></u>org</a><br>
<br>
Other people have developed this stuff. You're right, the patches are<br>
(by design) *not* kD-trees. To get kD-trees, you have to use the<br>
.tiles interface, like so:<br>
<br>
pf = load(...)<br>
data_source = pf.h.some_object_type(...)<br>
for (g, node, (sl, dims, gi)) in data_source.tiles.slice_<u></u>traverse():<br>
...<br>
<br>
here g is the grid object, node is the node of the kD-tree (for things<br>
like neighbor searching and the like), slice is the slice into the<br>
grid object that defines the node of the kD-tree, dims is the<br>
dimensions of the kD-tree brick, and gi is the left-edge in integers<br>
of the grid object. You can also do traverse, which just iterates and<br>
returns the partitioned grid data and which takesa viewpoint.<br>
<br>
-Matt<div><div class="h5"><br>
<br>
On Mon, Mar 10, 2014 at 1:45 PM, Alex Bogert <<a href="mailto:bogart.alex@gmail.com" target="_blank">bogart.alex@gmail.com</a>> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Matt,<br>
<br>
I remeber you saying yt can give us AMR in a kd-tree. Was this something<br>
like .get_grids or .get_tiles ?<br>
<br>
Thanks,<br>
Alex<br>
<br>
On Mar 10, 2014 10:21 AM, "John Holdener" <<a href="mailto:jrholden@ucsc.edu" target="_blank">jrholden@ucsc.edu</a>> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I'll be in lab shortly near 11. There is a serious complication that I<br>
need to talk to Matt about on the phone (when he gets time). I've been<br>
playing with patches, which are not kt-trees (there are many elements with<br>
no parent or children). This format would take forever searching every grid<br>
for samples.<br>
<br>
Once yt gives me a kt-tree, namely whatever class gives it to me, it<br>
should become just CUDA coding.<br>
<br>
Everything will need to be global memory, due to texture count constraints<br>
anyways. I suspect the first images will look awful (until some estimated<br>
interpolation happens). Until we get trees sorted out, AMR is at a<br>
standstill.<br>
<br>
<br>
<br>
Sent from my iPhone<br>
<br>
On Mar 10, 2014, at 10:04 AM, Alex Bogert <<a href="mailto:bogart.alex@gmail.com" target="_blank">bogart.alex@gmail.com</a>> wrote:<br>
<br>
Any update on AMR?<br>
<br>
On Mar 6, 2014 10:18 PM, "John Holdener" <<a href="mailto:jrholden@ucsc.edu" target="_blank">jrholden@ucsc.edu</a>> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I could buy that. However the number of typos typing Theia is going to<br>
drive me nuts!<br>
<br>
On 3/6/14, 3:03 PM, Alex Bogert wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Both Joel and I agree we need a good name for the software. I found<br>
"Theia" goddess of tracing and the origin of all light. The greeks believe<br>
she casted light from their eyes to see the world. Super literal<br>
encapsulation. Im open for suggestions! No more acronyms or anything to do<br>
with what languages we use. I think only programmers appreciate these type<br>
of names :)<br>
<br>
</blockquote>
--<br>
John R. Holdener<br>
Research Assistant<br>
UCSC HiPACC<br>
<a href="mailto:jrholden@ucsc.edu" target="_blank">jrholden@ucsc.edu</a><br>
<br>
</blockquote></blockquote></blockquote></div></div></blockquote><div><div class="h5">
<br>
-- <br>
John R. Holdener<br>
Research Assistant<br>
UCSC HiPACC<br>
<a href="mailto:jrholden@ucsc.edu" target="_blank">jrholden@ucsc.edu</a><br>
<br>
</div></div></div></div></div><br></div>
<br>_______________________________________________<br>
yt-dev mailing list<br>
<a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
<br></blockquote></div><br></div>