[yt-dev] Fwd: [yt_analysis/yt] Volume Rendering Refactor (pull request #168)

[Christopher Moody]

Hi Sam,
Pleiades had set OMP_NUM_THREADS=1, despite having specifically
requested a node with 12 OMP cores (ie, no HT). Nevertheless, the
overall efficiency is 14% having run projections continuously for 30
minutes. The kd-tree building doesn't scale, just stays a fixed
5-second cost, but the ray casting goes from ~60s to 8-9 seconds in
informal testing, so that part looks like it scales nicely on 12
cores. It looks like there's a lot of deadtime beween the kdtree and
ray-casting, and that that eats up most of wall clock time?

chris

On Wed, Jun 13, 2012 at 7:36 PM, Sam Skillman <samskillman at gmail.com> wrote:
> Hi Chris,
>
> Hmm,  maybe Pleiades sets OMP_NUM_THREADS=1 by default.  Can you check:
> echo $OMP_NUM_THREADS
>
> If it is either unset or set to 1, could you give it a shot with export
> OMP_NUM_THREADS=12?  The HT shouldn't help too much.
>
> Best,
> Sam
>
>
> On Wed, Jun 13, 2012 at 8:14 PM, Christopher Moody <cemoody at ucsc.edu> wrote:
>>
>> Hi Matt,
>> Yep, this is an ART dataset. I had a node running with 12 (24 with HT)
>> cores at NASA Pleiades. It didn't go above 4% (=1/24) usage, which
>> indicates means it likely did not  use more than one core (but I
>> didn't tell it to). If there's a parallel version of the script I can
>> try it out tonight, and see the scaling.
>>
>> chris
>>
>> On Wed, Jun 13, 2012 at 1:01 PM, Matthew Turk <matthewturk at gmail.com>
>> wrote:
>> > Chris, is this an ART dataset rendering?  I love it.  Great work.
>> > And, did you note on your machine what the load went up to?  Did it
>> > scale with the numebr of cores okay?
>> >
>> > On Wed, Jun 13, 2012 at 3:47 PM, Christopher Moody <cemoody at ucsc.edu>
>> > wrote:
>> >> Works for me!
>> >>
>> >> On one core, no_ghost=True, kd-Tree took 5 seconds, ray casting 56
>> >> seconds. Impressive.
>> >>
>> >> chris
>> >>
>> >> On Wed, Jun 13, 2012 at 10:45 AM, Matthew Turk <matthewturk at gmail.com>
>> >> wrote:
>> >>> Also, for what it's worth, John, Sam and I have been working on
>> >>> creating a movie sequence with this in HiDef for a display project.
>> >>> Once that's done we're going to be posting the movie, the original
>> >>> scripts, and potentially even a screencast of how one goes about doing
>> >>> a complex volume rendering.
>> >>>
>> >>> On Wed, Jun 13, 2012 at 12:34 PM, Sam Skillman <samskillman at gmail.com>
>> >>> wrote:
>> >>>> Hi all,
>> >>>>
>> >>>> Here's a sample script that exposes a few of the new features such as
>> >>>> opacity and the map_to_colormap function.
>> >>>> http://paste.yt-project.org/show/2458/
>> >>>>
>> >>>> I'll see if I can get some more scripts written in a portable way.
>> >>>>
>> >>>> Cheers,
>> >>>> Sam
>> >>>>
>> >>>> On Wed, Jun 13, 2012 at 10:22 AM, John ZuHone <jzuhone at gmail.com>
>> >>>> wrote:
>> >>>>>
>> >>>>> Matt,
>> >>>>>
>> >>>>> I'll give it a whirl... it may take a day or two.
>> >>>>>
>> >>>>> John Z
>> >>>>>
>> >>>>> On Jun 13, 2012, at 11:12 AM, Matthew Turk wrote:
>> >>>>>
>> >>>>> > Sam and I chatted, and we'd like to request three people test this
>> >>>>> > and
>> >>>>> > if they all agree, we can accept it.
>> >>>>> >
>> >>>>> > Can three people give it a go?
>> >>>>> >
>> >>>>> >
>> >>>>> > ---------- Forwarded message ----------
>> >>>>> > From: Matthew Turk <pullrequests-noreply at bitbucket.org>
>> >>>>> > Date: Wed, Jun 13, 2012 at 11:12 AM
>> >>>>> > Subject: [yt_analysis/yt] Volume Rendering Refactor (pull request
>> >>>>> > #168)
>> >>>>> > To: yt at enzotools.org
>> >>>>> >
>> >>>>> >
>> >>>>> > A new pull request has been opened by Matthew Turk.
>> >>>>> >
>> >>>>> > MatthewTurk/yt-refactor has changes to be pulled into
>> >>>>> > yt_analysis/yt.
>> >>>>> >
>> >>>>> >
>> >>>>> >
>> >>>>> > https://bitbucket.org/yt_analysis/yt/pull-request/168/volume-rendering-refactor
>> >>>>> >
>> >>>>> > Title: Volume Rendering Refactor
>> >>>>> >
>> >>>>> > This includes all of the changes to the volume rendering system
>> >>>>> > that
>> >>>>> > were discussed here:
>> >>>>> >
>> >>>>> >
>> >>>>> >
>> >>>>> > http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/2012-June/002039.html
>> >>>>> >
>> >>>>> > The main user-facing changes that could cause problems:
>> >>>>> >
>> >>>>> > 1) Cython 0.16 is now required.
>> >>>>> > 2) Off-axis projections no longer interpolate.
>> >>>>> >
>> >>>>> > Changes to be pulled:
>> >>>>> >
>> >>>>> >
>> >>>>> >
>> >>>>> > --
>> >>>>> > This is an issue notification from bitbucket.org.
>> >>>>> > You are receiving this either because you are the participating
>> >>>>> > in a pull request, or you are following it.
>> >>>>> > _______________________________________________
>> >>>>> > yt-dev mailing list
>> >>>>> > yt-dev at lists.spacepope.org
>> >>>>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>>>>
>> >>>>> _______________________________________________
>> >>>>> yt-dev mailing list
>> >>>>> yt-dev at lists.spacepope.org
>> >>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>>>
>> >>>>
>> >>>>
>> >>>> _______________________________________________
>> >>>> yt-dev mailing list
>> >>>> yt-dev at lists.spacepope.org
>> >>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>>>
>> >>> _______________________________________________
>> >>> yt-dev mailing list
>> >>> yt-dev at lists.spacepope.org
>> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>
>> >> _______________________________________________
>> >> yt-dev mailing list
>> >> yt-dev at lists.spacepope.org
>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>
>> > _______________________________________________
>> > yt-dev mailing list
>> > yt-dev at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>