[yt-dev] Rendering & Off-Axis Projection Changes: OpenMP, Opaque Renders, Lighting

Fri Jun 8 09:23:13 PDT 2012

Thanks, Nathan.  Once the initial refactoring into multiple kernels was
done, the testing was challenging.

There was one thing that your email reminded me of -- not only did we move
all the volume rendering stuff into the grid_traversal.pyx file (which
contains the samplers and stuff Sam discussed), but this fork *also* does
away with the amr_utils.pyx module in favor of many decoupled modules which
live under yt/utilities/lib .

It'll be pretty helpful for testing results to be sent here to the list.

On Fri, Jun 8, 2012 at 12:20 PM, Nathan Goldbaum <goldbaum at ucolick.org>wrote:

> Congratulations on getting this done and thanks for all the hard work Sam
> and Matt!  I look forward to testing this.
>
>
>
> On Jun 8, 2012, at 9:10 AM, Sam Skillman <samskillman at gmail.com> wrote:
>
> Hi all,
>
> Matt and I have been working a bunch in the volume_refactor bookmark in
> our yt-refactor forks and have added a bunch of functionality and
> performance improvements.  We are now entering a point where we will be
> issuing a PR soon.  At that point it should be tested by as many people as
> possible to ensure coverage.  I’m going to break this down into things that
> will change behavior and new things.
>
> The underlying code in the volume renderer has been refactored, simplified
> and streamlined.  Adding new types of volume rendering should now be much
> easier, since the act of walking the cells and the accumulation of values
> in the plane are now completely separated.
>
> --Things that will change in some way:
> *Off-axis Projections:
> We have implemented a new way of calculating off-axis projections which
> no-longer requires ghost zones and interpolation.  Instead, as the rays
> traverse the cells, they take the value directly from the value for that
> cell.  This leads to several changes.  First, it is way faster!  Second,
> because we aren’t interpolating anymore, that means you won’t get quite as
> smooth of an image as you may have before.  Below are before and after
> images.  Note that the transpose of the image is due to changes in the
> default behavior of setting up the looking vectors and the writing of the
> png in write_image.
>
> left: old  right: new
> **
>
> The performance gains for a sample (largish) dataset:
> Old: 186 seconds, including IO
> New (single-threaded): 20 seconds, including IO
>
> If you use 8 OpenMP threads (see later), the new drops down to 10 seconds,
> with an improvement from 18 to 8 in the section that is threaded.  Not
> perfect scaling, but there’s not a lot of work per cell at that point.
>
> *Isocontour/Other Renderings:
> These may change in their appearance slightly, mostly because of the
> changes made to allow for opaque and semi-opaque renderings.  If you were
> using transfer functions that have values close to 1.0 for the alpha, then
> things may start getting opaque.  More on this later.
>
> -- Things that are new:
> *OpenMP Threading:
> We’ve added support for OpenMP in some of the cython files such as
> grid_traversal.pyx.  This means that by default your renderings will run
> with OpenMP threads.  By default it will use the number of cores on
> whatever computer/node you are on. You can set this by either adjusting the
> environment variable OMP_NUM_THREADS or using the num_threads keyword in
> camera.snapshot().  Just be aware of it, and take a gander at top to see yt
> running at >100%!  This will hopefully allow users to run without mpi for
> rendering on their desktops/laptops.   Here’s a snapshot from John running
> on (presumably) 24 cores:
> *
> *
> **
> *
> *
> A side effect is that Cython 0.16 is now *required*.  This is included in
> the install_script.sh, which can be re-run at any time over top of an
> existing installation to update dependencies.
>
> *Opaque Rendering:
> Support for opaque rendering is now possible.  This is mostly enabled by a
> few changes to the allowed values in the transfer function.  The opacity is
> dictated by the values in the rgb of the transfer function.  This is a
> continuous behavior, so if you slowly ramp up the rgb values from 0.1 to
> 10, you’ll see it get more and more opaque.  In general it works out that
> rgb values around 1.0 start making things opaque, but you may need to
> experiment to get the desired behavior.  By the time 2.4 is released we
> plan on having a tutorial on how to make more and more complex renderings,
> starting from a simple rendering and moving to a more complicated example.
>  Currently the behavior is set so that all colors are opaque to all other
> colors.  The integration step (in
> yt/utilities/lib/field_interpolation_tables.pxd) is:
>     ttot = trgba[0] + trgba[1] + trgba[2]
>     ta = fmax(1.0 - dt*ttot, 0.0)
>     for i in range(3):
>         rgba[i] = (1.0-ta)*trgba[i] + ta*rgba[i]
> trgba is the transfer function rgba
> rgba is the image values
>
>
> *Lighting:
> We also now enable adding a light source to the rendering.  The behavior
> is enabled by use_light=True in the camera() call.  It is then controlled
> by cam.light_dir and cam.light_rgba.  The light direction is then dotted
> with the gradient of the field being rendered to either add or subtract
> color from that region.
> Here the integration is:
>     ta = fmax(1.0-dt*(trgba[0] + trgba[1] + trgba[2]), 0.0)
>     for i in range(3):
>         rgba[i] = (1.-ta)*trgba[i]*(1. + dot_prod*l_rgba[i]) + ta * rgba[i]
> where dot_prod is the (gradient of field) dot (light_direction), and
> l_rgba is the light color.
> What this does is amplify the emission when the dot product is positive,
> and decrease it when it is negative.
>
> So, what would be awesome is if you could try out your old scripts, and
> then try out some of the new features.  Remember, you may need to run the
> install script again to get an up-to-date cython.  There are two simple
> options to try this out.  The first is to pull the volume_refactor bookmark
> into your local repository.
>
> hg pull -B volume_refactor https://bitbucket.org/MatthewTurk/yt-refactor
> hg up volume_refactor
>
> This will create a head, which you shouldn’t worry about merging with any
> other head, with the bookmark volume_refactor.  Then a python setup.py
> develop or whatever you like should do it.
>
> If you don’t want to use bookmarks (though you really should because they
> are awesome!), then maybe just cloning the whole repo is for you.
>
>
> Once we let this simmer on yt-dev for a while, we will put up a PR to get
> this merged into the yt tip, and send out a compact version of this email
> to the yt-users list.  And yes, the PR will have lot of updated docs to go
> with it.  We are also working on a narrative example of how to go from a
> simple rendering like in the cookbook to something much more complex, and
> maybe even a screencast of one of us doing that in real time.
>
>
> Cheers,
> Sam & Matt
> !DSPAM:10175,4fd2241f258181156422526!
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
> !DSPAM:10175,4fd2241f258181156422526!
>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20120608/4c791864/attachment.html>