Matt,<div><br></div><div>This looks great so far. I'll take a look at how to get it going on the amr-kdtree homogenization and either send you a bundle or just wait till some of this gets pushed. </div><div><br></div>
<div>Sam</div><div><br></div><div><br><div class="gmail_quote">On Sun, Jan 9, 2011 at 7:28 PM, Matthew Turk <span dir="ltr"><<a href="mailto:matthewturk@gmail.com">matthewturk@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hi all,<br>
<br>
I've moved the discussion over to yt-dev, because I ended up writing<br>
up a very short set of changesets that test it out. This is the<br>
simplest implementation; it uses a kD-tree I grabbed off googlecode,<br>
which we'll ultimately want to replace with something faster, and for<br>
each cell it calculates all the stars to consider. At each sampling<br>
point it sums the contribution from all the stars (for now, they all<br>
produce pure white emission, distributed as a Gaussian) and then adds<br>
that to the incoming intensity for the purposes of computing the<br>
transfer function for the baryons. Note that currently it samples the<br>
Gaussian and does rectangular integration. (See below for more on<br>
this.) The effective radius is set to 1%, but as Greg mentioned on<br>
yt-users that can probably be made more loose.<br>
<br>
There are several problems, which I was hoping to work out before I<br>
sent this, but in the interest of getting it out there for someone<br>
else to possibly fix up I thought I'd share what I had first.<br>
<br>
1) There are grid artifacts. I tried tracking these down, but I was<br>
unable to do so. I think it may be related to cell center/vertex<br>
center data.<br>
2) It's pretty slow, but my profiling shows that as showing up in the<br>
retrieving of data, not in the searching of the kD-tree. This may be<br>
a mistake in my reading of the profiles. It's slowest for images<br>
where a single star contributes to many pixels, because of the way the<br>
loops are nested.<br>
3) It assumes a uniform sigma and a uniform emission coefficient<br>
(equal in all three channels, i.e. white light)<br>
4) It only works with homogenized volume; I don't understand how<br>
PartitionedGrid objects are assigned to the kD-tree so I couldn't use<br>
that.<br>
<br>
One definite improvement that will need to happen is to remove the<br>
direct calculation and sampling of the Gaussian inside the function<br>
add_stars in yt/utilities/_amr_utils/VolumeIntegrator.pyx. The<br>
sampling as a mechanism for integration (either rectangular or<br>
trapezoidal) is going to miss points if your step size internal to a<br>
cell could step over the centroid of the Gaussian. I believe the<br>
right way to solve this is to cease calculating direct radius from the<br>
sampling point to the Gaussian, and instead decompose the Gaussian<br>
into the cylindrical radius (impact parameter) component and the<br>
component along the ray. You then would use a tabulated erf(t)<br>
function to get the total contribution to the intensity over t' to t'<br>
+ dt. This should be better quality, although it may end up being a<br>
wash for speed. This would also help to ensure that the peak of any<br>
given Gaussian doesn't get skipped during the integration.<br>
<br>
I've stuck the bundle up here:<br>
<br>
<a href="http://yt.enzotools.org/files/star_rendering.hg" target="_blank">http://yt.enzotools.org/files/star_rendering.hg</a><br>
<br>
You'll have to re-cythonize yt/utilities/amr_utils.pyx and all the<br>
changes have been made to<br>
yt/utilities/_amr_utils/VolumeIntegrator.pyx. The function add_stars<br>
is the main one.<br>
<br>
A sample script is here:<br>
<br>
<a href="http://paste.enzotools.org/show/1481" target="_blank">http://paste.enzotools.org/show/1481</a><br>
<br>
and I've attached two images of the Enzo Workshop sample dataset<br>
JHK-DD0030, provided by Ji-hoon Kim. The first (raw_stars.png) is<br>
what the image looks like straight out of the code. The second one<br>
I've compressed the color scale, so you can see where the star<br>
particles all are. You can definitely see the grid artifacts in both!<br>
<br>
Anyway, if you play with it, let me know what you think. Especially<br>
if you can figure out the grid boundary thing! :)<br>
<br>
-Matt<br>
<div><div></div><div class="h5"><br>
<br>
On Fri, Jan 7, 2011 at 3:13 PM, j s oishi <<a href="mailto:jsoishi@gmail.com">jsoishi@gmail.com</a>> wrote:<br>
> As a note, that's what Orion does for all its particles, and it works just fine.<br>
><br>
> On Fri, Jan 7, 2011 at 12:01 PM, John Wise <<a href="mailto:jwise@astro.princeton.edu">jwise@astro.princeton.edu</a>> wrote:<br>
>> Hi Matt,<br>
>><br>
>> This all sounds great. I like the idea of associating stars with bricks to simplify the search.<br>
>><br>
>> I think it's the easiest and best approach now (maybe not at petascale) to have all star particles duplicated on each processor. I can't think of any simulation with more than a few million star particles, and that easily fits into memory. This is the same approach I've taken with the new star particles in Enzo. I thought it would be best to exploit the fact that the problem wasn't memory limited.<br>
>><br>
>> My 2c.<br>
>> John<br>
>><br>
>> On 6 Jan 2011, at 18:30, Matthew Turk wrote:<br>
>><br>
>>> Hi John,<br>
>>><br>
>>> (As a quick comment, one can export to Sunrise from yt, so that could<br>
>>> also serve as a mechanism for rendering star particles.)<br>
>>><br>
>>> I have been thinking about this a lot lately, and I think you're<br>
>>> right: we need a proper mechanism for compositing star particles on<br>
>>> the fly during the traversal of rays across a homogenized volume. I<br>
>>> had planned on this being one of my first yt projects this year. The<br>
>>> current process of volume rendering (for more detail see the method<br>
>>> paper) is basically:<br>
>>><br>
>>> 1) Homogenize volume, splitting fields up into uniquely-tiling bricks<br>
>>> 2) Sort bricks<br>
>>> 3) For every brick, for every ray:<br>
>>> a) Calculate intersection<br>
>>> b) Update all channels (typically 3) based on *local* emission and absorption<br>
>>> c) Update ray position<br>
>>> 4) Return image plane<br>
>>><br>
>>> This is true for both the old, homogenized volume rendering technique<br>
>>> and the new kD-tree technique. To fit star particles into this, we<br>
>>> would regard them as exclusively sources of emission, with no impact<br>
>>> on the local absorption. Nominally, this should be easy to do: for<br>
>>> every cell, simply deposit the emission from a star. As you noted in<br>
>>> your email, this results in very, very ugly results -- I tested it<br>
>>> last summer with the hopes of coming up with something cool, but was<br>
>>> unable to. Testing it today on an airplane showed it had bitrot a<br>
>>> bit, so I haven't attached it. :)<br>
>>><br>
>>> I think we would need to move to, rather than cell-based emission (so<br>
>>> that the smallest emission point from a star is a single cell) to a<br>
>>> method where emission from star particles is calculated per ray (i.e.,<br>
>>> pixel). This would require an additional step:<br>
>>><br>
>>> 1) Homogenize volume, splitting fields up into uniquely-tiling bricks<br>
>>> 2) Sort bricks<br>
>>> 3) For every brick, for every ray:<br>
>>> a) Calculate intersection<br>
>>> b) Calculate emission from stars local to the ray<br>
>>> c) Update all channels (typically 3) based on *local* emission and absorption<br>
>>> d) Update ray position<br>
>>> 4) Return image plane<br>
>>><br>
>>> This would enable both density-based emission *and* star emission.<br>
>>> This could be both density isocontours, for instance, and individual<br>
>>> stars. The visual language in that would probably be very confusing,<br>
>>> but it would be possible, particularly for pure annotations.<br>
>>><br>
>>> The issue here is that step 2b is probably very, very slow -- even<br>
>>> using a (point-based) kD-tree it would likely add substantial run<br>
>>> time, because there's no early termination mechanism. What I think we<br>
>>> could do, however, is execute a pre-deposition phase. For the<br>
>>> purposes of rendering, we can describe a star particle by only a few<br>
>>> characteristics:<br>
>>><br>
>>> Emission(Red, Green, Blue)<br>
>>> Gaussian(Radius)<br>
>>> Position(x,y,z)<br>
>>><br>
>>> We should instead define an effective radius (ER), say at the 1%<br>
>>> level, at which we won't worry anymore. We could then deposit delta<br>
>>> functions of size ER for every star particle. This would give a cue<br>
>>> to the ray caster, and we could modify:<br>
>>><br>
>>> 1) Homogenize volume, splitting fields up into uniquely-tiling bricks<br>
>>> 2) Sort bricks<br>
>>> 3) For every brick, for every ray:<br>
>>> a) Calculate intersection<br>
>>> b) If local delta_field == True, execute ball query and calculate<br>
>>> emission from stars local to the ray<br>
>>> c) Update all channels (typically 3) based on *local* emission and absorption<br>
>>> d) Update ray position<br>
>>> 4) Return image plane<br>
>>><br>
>>> For the first pass, we would probably want all our stars to have the<br>
>>> same ER, which would then be the radius of our ball-query. For<br>
>>> parallel rendering, we would still have to have all of the star<br>
>>> particles loaded on every processor; I don't think this is a problem,<br>
>>> since in the limit where the star particles are memory-limiting, you<br>
>>> would likely not suffer from pre-deposition. This also solves the<br>
>>> grid-boundary issues, as each processor would deposit all stars during<br>
>>> its initial homogenization.<br>
>>><br>
>>> What do you think? I think that the components external to the ray<br>
>>> tracer could be assembled relatively easily, and then the ray tracer<br>
>>> might take a bit of work. As a post-processing step we could even add<br>
>>> lens flare, for that extra Star Trek look.<br>
>>><br>
>>> -Matt<br>
>>><br>
>>><br>
>>> On Thu, Jan 6, 2011 at 8:45 AM, John Wise <<a href="mailto:jwise@astro.princeton.edu">jwise@astro.princeton.edu</a>> wrote:<br>
>>>> I forgot to mention that another way to do this is making a derived field<br>
>>>> that adds the stellar density to the gas density. However this doesn't look<br>
>>>> good when particles are in coarse grids, when they should be point sources<br>
>>>> in the image.<br>
>>>><br>
>>>> def _RelativeDensityStars(field, data):<br>
>>>> return (data["Density"] + data["star_density"])/dma<br>
>>>> add_field("RelativeDensityStars", function=_RelativeDensityStars,<br>
>>>> take_log = False)<br>
>>>><br>
>>>> where dma is a scaling variable.<br>
>>>><br>
>>>> I'm uploading my stand-alone script if you want to try to decipher it,<br>
>>>> although I tried to comment it some.<br>
>>>><br>
>>>> <a href="http://paste.enzotools.org/show/1475/" target="_blank">http://paste.enzotools.org/show/1475/</a><br>
>>>><br>
>>>> Also I uploaded the colormap based on B-V colors that I ripped from<br>
>>>> partiview to<br>
>>>><br>
>>>> <a href="http://www.astro.princeton.edu/~jwise/temp/BminusV.h5" target="_blank">http://www.astro.princeton.edu/~jwise/temp/BminusV.h5</a><br>
>>>><br>
>>>> John<br>
>>>><br>
>>>> On 01/06/2011 11:14 AM, John Wise wrote:<br>
>>>>><br>
>>>>> Hi Libby,<br>
>>>>><br>
>>>>> I'm afraid that there isn't a good solution for rendering stars, at<br>
>>>>> least to my knowledge!<br>
>>>>><br>
>>>>> You can add them as pixels after you've determined the pixel numbers (as<br>
>>>>> in Andrew's email) of the particles with the splat_points() routine in<br>
>>>>> the image_writer module.<br>
>>>>><br>
>>>>> I wrote my own stand-alone splatter to put Gaussian splats for<br>
>>>>> particles, but I never incorporated it into yt. I meant to a few months<br>
>>>>> back when I wrote it but never did! It will produce these types of splats<br>
>>>>><br>
>>>>><br>
>>>>> <a href="http://www.astro.princeton.edu/~jwise/research/GalaxyBirth_files/combine.png" target="_blank">http://www.astro.princeton.edu/~jwise/research/GalaxyBirth_files/combine.png</a><br>
>>>>><br>
>>>>><br>
>>>>> I had to manually blend the gas volume rendering and star splats<br>
>>>>> afterwards to produce that image.<br>
>>>>><br>
>>>>> I hope I can make something that looks as good as partiview soon. This<br>
>>>>> is the same dataset but with partiview.<br>
>>>>><br>
>>>>><br>
>>>>> <a href="http://www.astro.princeton.edu/~jwise/research/GalaxyBirth_files/stars_only.png" target="_blank">http://www.astro.princeton.edu/~jwise/research/GalaxyBirth_files/stars_only.png</a><br>
>>>>><br>
>>>>><br>
>>>>> I'll see if I can make time (first I have to find the code!) to<br>
>>>>> incorporate my splatter into yt.<br>
>>>>><br>
>>>>> John<br>
>>>>><br>
>>>>> On 01/06/2011 09:15 AM, Elizabeth Harper-Clark wrote:<br>
>>>>>><br>
>>>>>> Hi all,<br>
>>>>>><br>
>>>>>> Thanks for all your help over the last couple of days. One more question:<br>
>>>>>> - Can I plot particles on a volume rendered image?<br>
>>>>>> I have stars and I want to show where they are!<br>
>>>>>><br>
>>>>>> Thanks,<br>
>>>>>><br>
>>>>>> Libby<br>
>>>>>><br>
>>>>>> --<br>
>>>>>> Elizabeth Harper-Clark MA MSci<br>
>>>>>> PhD Candidate, Canadian Institute for Theoretical Astrophysics, UofT<br>
>>>>>> Sciences and Engineering Coordinator, Teaching Assistants' Training<br>
>>>>>> Program, UofT<br>
>>>>>><br>
>>>>>> <a href="http://www.astro.utoronto.ca/~h-clark" target="_blank">www.astro.utoronto.ca/~h-clark</a> <<a href="http://www.astro.utoronto.ca/%7Eh-clark" target="_blank">http://www.astro.utoronto.ca/%7Eh-clark</a>><br>
>>>>>> <a href="mailto:h-clark@cita.utoronto.ca">h-clark@cita.utoronto.ca</a> <mailto:<a href="mailto:h-clark@cita.utoronto.ca">h-clark@cita.utoronto.ca</a>><br>
>>>>>> Astronomy office phone: +1-416-978-5759<br>
>>>>>><br>
>>>>>><br>
>>>>>><br>
>>>>>> _______________________________________________<br>
>>>>>> yt-users mailing list<br>
>>>>>> <a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
>>>>>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
>>>>><br>
>>>>> _______________________________________________<br>
>>>>> yt-users mailing list<br>
>>>>> <a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
>>>>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
>>>><br>
>>>> _______________________________________________<br>
>>>> yt-users mailing list<br>
>>>> <a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
>>>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
>>>><br>
>>> _______________________________________________<br>
>>> yt-users mailing list<br>
>>> <a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
>>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
>><br>
>> _______________________________________________<br>
>> yt-users mailing list<br>
>> <a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
>><br>
> _______________________________________________<br>
> yt-users mailing list<br>
> <a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
> <a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
><br>
</div></div><br>_______________________________________________<br>
Yt-dev mailing list<br>
<a href="mailto:Yt-dev@lists.spacepope.org">Yt-dev@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
<br></blockquote></div><br></div>