<p>Hi Sam,</p>
<p>Thanks for doing this. </p>
<p>I'll update my PR shortly; this needs to be a part of it before it's accepted.</p>
<p>This has also highlighted a part of the process that I think we're all trying to do -- peer review not just of the quantitative changes to the code (i.e., breaking tests!) but also qualitative. In this case, Cameron saw something that would change functionality, and he stood up for keeping the ability to retain backwards compatibility. I think this is a valid point and a valuable position; in the case of the off-axis projection, when I initially made this change it was because I felt this was truer to the underlying data. (There are extremely few places in the yt codebase where it interpolates.)</p>
<p>However, retaining a solid foundation for users is also extremely important. The growth of yt over the last couple years has put us in a sticky situation; we've developed a community, but it's a community that relies on functionality not changing without good reason, and more importantly that relies on keeping a solid basis for repeatable analysis and consistent results. This change broke that -- even though it's arguably an improvement of the solution -- and the old method was no longer available.</p>
<p>I'm really happy that this system has worked; while I was initially -1 on restoring interpolation, Cameron swayed me in IRC by bringing up this remarkably important point. I think turning the default to non-interpolated but retaining the interpolated mode are both the right decisions, and I'm +1. And more than that, I'm glad this all happened before the PR was accepted.</p>
<p>So thank you, Sam, and Cameron, for restoring the old functionality. I'm excited about moving forward. The results in this thread have really gotten me excited about the success of this new set of changes.</p>
<p>-Matt</p>
<div class="gmail_quote">On Jun 13, 2012 11:26 PM, "Sam Skillman" <<a href="mailto:samskillman@gmail.com" target="_blank">samskillman@gmail.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi all,<div><br></div><div>Earlier today Cameron brought up some concerns about removing the behavior of the off-axis-projection using interpolated data from vertex-centered data. If you recall from earlier discussions on the impact of the new threading + opaque renderings + refactoring, one of the things we've done is switch from the interpolation to using the raw values within each cell a given ray traverses for use in the off-axis-projection. This is mathematically a more accurate representation of the data, but can sometimes lead to blocky looking images. </div>
<div><br></div><div>What I've done is go through and add an optional interpolated=(False/True) argument to the off_axis_projection call, where the default is False. Additionally, I've set this up so that it now wraps the ProjectionCamera class, which is derived from the Camera class, so it has things like .rotate, etc. </div>
<div><br></div><div>Just for reference, I'm including below both timings, results, and example images, made using this script:</div><div><a href="http://paste.yt-project.org/show/2462/" target="_blank">http://paste.yt-project.org/show/2462/</a></div>
<div><br></div><div>Here is the output:</div><div><div>Comparison of Raw vs Interpolated</div><div>Density</div><div>Raw -- Time: 1.173431e+00 seconds, min: 8.046089e-06, max: 1.177795e-02</div><div>Int -- Time: 6.393837e+00 seconds, min: 8.322453e-06, max: 1.113421e-02</div>
<div><br></div><div>Density-weighted Temperature</div><div>Raw -- Time: 8.513930e-01 seconds, min: 4.681834e+00, max: 1.237399e+07</div><div>Int -- Time: 8.450805e+00 seconds, min: 1.105447e+01, max: 1.213922e+07</div></div>
<div><br></div><div>The main difference that you see here is that the interpolated doesn't capture the minima and maxima as well, which makes sense since it is interpolating the data and smoothing out the peaks. Note that the Interpolated use the no_ghost=False option, which brings it much closer to the Raw values. This highlights the improvement off the off_axis_projection for quantitative results, as it is both more accurate and faster. This is also a fairly small dataset. Improvements for larger simulations are even more impressive. </div>
<div><br></div><div>You can see the diff between this changeset and Matt's changeset that is currently the PR changeset here:</div><div><a href="https://bitbucket.org/samskillman/yt-refactor/compare/..MatthewTurk/yt-refactor" target="_blank">https://bitbucket.org/samskillman/yt-refactor/compare/..MatthewTurk/yt-refactor</a></div>
<div><br></div><div>Please take a look. I'm hopeful that with these changes pulled into Matt's PR, we should be closer to this being accepted.</div><div><br></div><div>Along with these changes come some updated docs, as well as getting rid of those pesky print statements that Britton saw.</div>
<div><br></div><div>Cheers,</div><div>Sam</div><div><br></div><br><div class="gmail_quote">On Wed, Jun 13, 2012 at 8:36 PM, Sam Skillman <span dir="ltr"><<a href="mailto:samskillman@gmail.com" target="_blank">samskillman@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>Hi Chris,</div><div><br></div>Hmm, maybe Pleiades sets OMP_NUM_THREADS=1 by default. Can you check:<div>echo $OMP_NUM_THREADS</div>
<div><br></div><div>If it is either unset or set to 1, could you give it a shot with export OMP_NUM_THREADS=12? The HT shouldn't help too much. </div>
<div><br></div><div>Best,</div><div>Sam</div><div><div><div><div><br><br><div class="gmail_quote">On Wed, Jun 13, 2012 at 8:14 PM, Christopher Moody <span dir="ltr"><<a href="mailto:cemoody@ucsc.edu" target="_blank">cemoody@ucsc.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Matt,<br>
Yep, this is an ART dataset. I had a node running with 12 (24 with HT)<br>
cores at NASA Pleiades. It didn't go above 4% (=1/24) usage, which<br>
indicates means it likely did not use more than one core (but I<br>
didn't tell it to). If there's a parallel version of the script I can<br>
try it out tonight, and see the scaling.<br>
<br>
chris<br>
<div><br>
On Wed, Jun 13, 2012 at 1:01 PM, Matthew Turk <<a href="mailto:matthewturk@gmail.com" target="_blank">matthewturk@gmail.com</a>> wrote:<br>
</div><div><div>> Chris, is this an ART dataset rendering? I love it. Great work.<br>
> And, did you note on your machine what the load went up to? Did it<br>
> scale with the numebr of cores okay?<br>
><br>
> On Wed, Jun 13, 2012 at 3:47 PM, Christopher Moody <<a href="mailto:cemoody@ucsc.edu" target="_blank">cemoody@ucsc.edu</a>> wrote:<br>
>> Works for me!<br>
>><br>
>> On one core, no_ghost=True, kd-Tree took 5 seconds, ray casting 56<br>
>> seconds. Impressive.<br>
>><br>
>> chris<br>
>><br>
>> On Wed, Jun 13, 2012 at 10:45 AM, Matthew Turk <<a href="mailto:matthewturk@gmail.com" target="_blank">matthewturk@gmail.com</a>> wrote:<br>
>>> Also, for what it's worth, John, Sam and I have been working on<br>
>>> creating a movie sequence with this in HiDef for a display project.<br>
>>> Once that's done we're going to be posting the movie, the original<br>
>>> scripts, and potentially even a screencast of how one goes about doing<br>
>>> a complex volume rendering.<br>
>>><br>
>>> On Wed, Jun 13, 2012 at 12:34 PM, Sam Skillman <<a href="mailto:samskillman@gmail.com" target="_blank">samskillman@gmail.com</a>> wrote:<br>
>>>> Hi all,<br>
>>>><br>
>>>> Here's a sample script that exposes a few of the new features such as<br>
>>>> opacity and the map_to_colormap function.<br>
>>>> <a href="http://paste.yt-project.org/show/2458/" target="_blank">http://paste.yt-project.org/show/2458/</a><br>
>>>><br>
>>>> I'll see if I can get some more scripts written in a portable way.<br>
>>>><br>
>>>> Cheers,<br>
>>>> Sam<br>
>>>><br>
>>>> On Wed, Jun 13, 2012 at 10:22 AM, John ZuHone <<a href="mailto:jzuhone@gmail.com" target="_blank">jzuhone@gmail.com</a>> wrote:<br>
>>>>><br>
>>>>> Matt,<br>
>>>>><br>
>>>>> I'll give it a whirl... it may take a day or two.<br>
>>>>><br>
>>>>> John Z<br>
>>>>><br>
>>>>> On Jun 13, 2012, at 11:12 AM, Matthew Turk wrote:<br>
>>>>><br>
>>>>> > Sam and I chatted, and we'd like to request three people test this and<br>
>>>>> > if they all agree, we can accept it.<br>
>>>>> ><br>
>>>>> > Can three people give it a go?<br>
>>>>> ><br>
>>>>> ><br>
>>>>> > ---------- Forwarded message ----------<br>
>>>>> > From: Matthew Turk <<a href="mailto:pullrequests-noreply@bitbucket.org" target="_blank">pullrequests-noreply@bitbucket.org</a>><br>
>>>>> > Date: Wed, Jun 13, 2012 at 11:12 AM<br>
>>>>> > Subject: [yt_analysis/yt] Volume Rendering Refactor (pull request #168)<br>
>>>>> > To: <a href="mailto:yt@enzotools.org" target="_blank">yt@enzotools.org</a><br>
>>>>> ><br>
>>>>> ><br>
>>>>> > A new pull request has been opened by Matthew Turk.<br>
>>>>> ><br>
>>>>> > MatthewTurk/yt-refactor has changes to be pulled into yt_analysis/yt.<br>
>>>>> ><br>
>>>>> ><br>
>>>>> > <a href="https://bitbucket.org/yt_analysis/yt/pull-request/168/volume-rendering-refactor" target="_blank">https://bitbucket.org/yt_analysis/yt/pull-request/168/volume-rendering-refactor</a><br>
>>>>> ><br>
>>>>> > Title: Volume Rendering Refactor<br>
>>>>> ><br>
>>>>> > This includes all of the changes to the volume rendering system that<br>
>>>>> > were discussed here:<br>
>>>>> ><br>
>>>>> ><br>
>>>>> > <a href="http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/2012-June/002039.html" target="_blank">http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/2012-June/002039.html</a><br>
>>>>> ><br>
>>>>> > The main user-facing changes that could cause problems:<br>
>>>>> ><br>
>>>>> > 1) Cython 0.16 is now required.<br>
>>>>> > 2) Off-axis projections no longer interpolate.<br>
>>>>> ><br>
>>>>> > Changes to be pulled:<br>
>>>>> ><br>
>>>>> ><br>
>>>>> ><br>
>>>>> > --<br>
>>>>> > This is an issue notification from <a href="http://bitbucket.org" target="_blank">bitbucket.org</a>.<br>
>>>>> > You are receiving this either because you are the participating<br>
>>>>> > in a pull request, or you are following it.<br>
>>>>> > _______________________________________________<br>
>>>>> > yt-dev mailing list<br>
>>>>> > <a href="mailto:yt-dev@lists.spacepope.org" target="_blank">yt-dev@lists.spacepope.org</a><br>
>>>>> > <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>>>>><br>
>>>>> _______________________________________________<br>
>>>>> yt-dev mailing list<br>
>>>>> <a href="mailto:yt-dev@lists.spacepope.org" target="_blank">yt-dev@lists.spacepope.org</a><br>
>>>>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>>>><br>
>>>><br>
>>>><br>
>>>> _______________________________________________<br>
>>>> yt-dev mailing list<br>
>>>> <a href="mailto:yt-dev@lists.spacepope.org" target="_blank">yt-dev@lists.spacepope.org</a><br>
>>>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>>>><br>
>>> _______________________________________________<br>
>>> yt-dev mailing list<br>
>>> <a href="mailto:yt-dev@lists.spacepope.org" target="_blank">yt-dev@lists.spacepope.org</a><br>
>>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>><br>
>> _______________________________________________<br>
>> yt-dev mailing list<br>
>> <a href="mailto:yt-dev@lists.spacepope.org" target="_blank">yt-dev@lists.spacepope.org</a><br>
>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
>><br>
> _______________________________________________<br>
> yt-dev mailing list<br>
> <a href="mailto:yt-dev@lists.spacepope.org" target="_blank">yt-dev@lists.spacepope.org</a><br>
> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
_______________________________________________<br>
yt-dev mailing list<br>
<a href="mailto:yt-dev@lists.spacepope.org" target="_blank">yt-dev@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
</div></div></blockquote></div><br></div></div>
</div></div></blockquote></div><br>
<br>_______________________________________________<br>
yt-dev mailing list<br>
<a href="mailto:yt-dev@lists.spacepope.org" target="_blank">yt-dev@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
<br></blockquote></div>