[yt-dev] Rendering & Off-Axis Projection Changes: OpenMP, Opaque Renders, Lighting

[Sam Skillman]

Hi all,

Matt and I have been working a bunch in the volume_refactor bookmark in our
yt-refactor forks and have added a bunch of functionality and performance
improvements.  We are now entering a point where we will be issuing a PR
soon.  At that point it should be tested by as many people as possible to
ensure coverage.  I’m going to break this down into things that will change
behavior and new things.

The underlying code in the volume renderer has been refactored, simplified
and streamlined.  Adding new types of volume rendering should now be much
easier, since the act of walking the cells and the accumulation of values
in the plane are now completely separated.

--Things that will change in some way:
*Off-axis Projections:
We have implemented a new way of calculating off-axis projections which
no-longer requires ghost zones and interpolation.  Instead, as the rays
traverse the cells, they take the value directly from the value for that
cell.  This leads to several changes.  First, it is way faster!  Second,
because we aren’t interpolating anymore, that means you won’t get quite as
smooth of an image as you may have before.  Below are before and after
images.  Note that the transpose of the image is due to changes in the
default behavior of setting up the looking vectors and the writing of the
png in write_image.

left: old  right: new
**

The performance gains for a sample (largish) dataset:
Old: 186 seconds, including IO
New (single-threaded): 20 seconds, including IO

If you use 8 OpenMP threads (see later), the new drops down to 10 seconds,
with an improvement from 18 to 8 in the section that is threaded.  Not
perfect scaling, but there’s not a lot of work per cell at that point.

*Isocontour/Other Renderings:
These may change in their appearance slightly, mostly because of the
changes made to allow for opaque and semi-opaque renderings.  If you were
using transfer functions that have values close to 1.0 for the alpha, then
things may start getting opaque.  More on this later.

-- Things that are new:
*OpenMP Threading:
We’ve added support for OpenMP in some of the cython files such as
grid_traversal.pyx.  This means that by default your renderings will run
with OpenMP threads.  By default it will use the number of cores on
whatever computer/node you are on. You can set this by either adjusting the
environment variable OMP_NUM_THREADS or using the num_threads keyword in
camera.snapshot().  Just be aware of it, and take a gander at top to see yt
running at >100%!  This will hopefully allow users to run without mpi for
rendering on their desktops/laptops.   Here’s a snapshot from John running
on (presumably) 24 cores:
*
*
**
*
*
A side effect is that Cython 0.16 is now *required*.  This is included in
the install_script.sh, which can be re-run at any time over top of an
existing installation to update dependencies.

*Opaque Rendering:
Support for opaque rendering is now possible.  This is mostly enabled by a
few changes to the allowed values in the transfer function.  The opacity is
dictated by the values in the rgb of the transfer function.  This is a
continuous behavior, so if you slowly ramp up the rgb values from 0.1 to
10, you’ll see it get more and more opaque.  In general it works out that
rgb values around 1.0 start making things opaque, but you may need to
experiment to get the desired behavior.  By the time 2.4 is released we
plan on having a tutorial on how to make more and more complex renderings,
starting from a simple rendering and moving to a more complicated example.
 Currently the behavior is set so that all colors are opaque to all other
colors.  The integration step (in
yt/utilities/lib/field_interpolation_tables.pxd) is:
    ttot = trgba[0] + trgba[1] + trgba[2]
    ta = fmax(1.0 - dt*ttot, 0.0)
    for i in range(3):
        rgba[i] = (1.0-ta)*trgba[i] + ta*rgba[i]
trgba is the transfer function rgba
rgba is the image values


*Lighting:
We also now enable adding a light source to the rendering.  The behavior is
enabled by use_light=True in the camera() call.  It is then controlled by
cam.light_dir and cam.light_rgba.  The light direction is then dotted with
the gradient of the field being rendered to either add or subtract color
from that region.
Here the integration is:
    ta = fmax(1.0-dt*(trgba[0] + trgba[1] + trgba[2]), 0.0)
    for i in range(3):
        rgba[i] = (1.-ta)*trgba[i]*(1. + dot_prod*l_rgba[i]) + ta * rgba[i]
where dot_prod is the (gradient of field) dot (light_direction), and l_rgba
is the light color.
What this does is amplify the emission when the dot product is positive,
and decrease it when it is negative.

So, what would be awesome is if you could try out your old scripts, and
then try out some of the new features.  Remember, you may need to run the
install script again to get an up-to-date cython.  There are two simple
options to try this out.  The first is to pull the volume_refactor bookmark
into your local repository.

hg pull -B volume_refactor https://bitbucket.org/MatthewTurk/yt-refactor
hg up volume_refactor

This will create a head, which you shouldn’t worry about merging with any
other head, with the bookmark volume_refactor.  Then a python setup.py
develop or whatever you like should do it.

If you don’t want to use bookmarks (though you really should because they
are awesome!), then maybe just cloning the whole repo is for you.


Once we let this simmer on yt-dev for a while, we will put up a PR to get
this merged into the yt tip, and send out a compact version of this email
to the yt-users list.  And yes, the PR will have lot of updated docs to go
with it.  We are also working on a narrative example of how to go from a
simple rendering like in the cookbook to something much more complex, and
maybe even a screencast of one of us doing that in real time.


Cheers,
Sam & Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20120608/18be2e33/attachment.htm>