[yt-users] yt volume render/AMRKDtree help

Mon Jan 14 12:26:32 PST 2013

Hi Renyue,

On Mon, Jan 14, 2013 at 3:23 PM, Renyue Cen <cen at astro.princeton.edu> wrote:
> Hi Sam,
>
> Constructing KDTree for the whole box apparently take a much much longer
> time
> (first it ran  out of memory so I increased the RAM/CPU ratio): it used to
> take about 1 min
> and the test run I put in has taken 80 mins and is still in the KDTree
> building phase,
> so it might not be practical for me with this approach.

The kD-tree is instrumented to operate in parallel; are you running
this in parallel?  For very large domains, I am not surprised it would
take considerably longer to do the whole domain.

Did you try Sam's suggestion of specifying LE/RE?  That would likely
be the best solution.

-Matt

>
> Best,
> Renyue
>
> On Jan 14, 2013, at 10:01 AM, Sam Skillman wrote:
>
> Hi Renyue,
>
> I've been travelling all over the place and have had flaky (at best)
> internet, so sorry for the delay.
>
> Unfortunately that error means that the AMRKDTree is failing for some of
> it's constructions, which is a fairly difficult problem to track down.  The
> most robust way to fix the problem is to build the full volume and even
> though it will be much slower, it will fix the problem.  There are
> improvements in an outstanding pull request for the new AMRKDTree that will
> fix this, but for now (until I'm done travelling at the end of this week)
> the best solution is to just hand it the full volume.  Note that if you can
> automate through each of the galaxies and only build the volume once, you
> can then switch the camera positions to go look around at all the galaxies.
>
> The only other option I can think of is to force le= and re= equal to root
> grid cell boundaries.  So if you had your original LE= c-0.5*WW and RE =
> c+0.5*WW
> Then I would do:
> le = np.int(LE*pf.domain_dimensions) * pf.domain_width /
> pf.domain_dimensions
> re = (np.int(RE*pf.domain_dimensions)+1) * pf.domain_width /
> pf.domain_dimensions
>
> I'm not positive that it will fix the problem, so if that doesn't work then
> please go back to the full domain (not specifying le, re).  Sorry for the
> trouble.
>
> Sam
>
>
> On Mon, Jan 14, 2013 at 6:47 AM, Renyue Cen <cen at astro.princeton.edu> wrote:
>>
>> Hi Sam,
>>
>> This problem that I mentioned in the attached email sent earlier appears
>> to be
>> a bit more common that I thought. I mean, it occurs for multiple galaxies
>> at
>> different redshifts. The data files appear ok in that I could make
>> projection plots
>> in the same regions just fine without using the AMTKDTree, and a same data
>> file
>> looks ok for AMRKDTree for a different galaxy at a different location.
>>
>> This is a bit critical for me because I need to be able to analyze all
>> these galaxies
>> to gain a full physical understanding.
>>
>> Your help is appreciated,
>>
>> Best,
>> Renyue
>>
>>
>> On Jan 9, 2013, at 10:57 PM, Renyue Cen wrote:
>>
>> Hi Sam,
>>
>> I have succeeded in volume rendering regions around some galaxies with the
>> transpose=false fix.
>> But, occasionally, for some reason, the exact same piece of yt code on
>> some galaxies
>> failed with the following error:
>>
>> AttributeError: 'NoneType' object has no attribute 'id'
>> Building kd-Tree  0% |                                        | ETA:
>> --:--:-- ^MTraceback (most recent call last):
>>   File "test.py", line 318, in <module>
>>     le=c-0.5*WW, re=c+0.5*WW)
>>   File
>> "/u/jhwise/local/lib/python2.7/site-packages/yt-2.4-py2.7-linux-x86_64.egg/yt/utilities/amr_kdtree/amr_kdtree.py",
>> line 376, in __init__
>>     self._build(root_grids, None, self.domain_left_edge,
>> self.domain_right_edge)
>>   File
>> "/u/jhwise/local/lib/python2.7/site-packages/yt-2.4-py2.7-linux-x86_64.egg/yt/utilities/amr_kdtree/amr_kdtree.py",
>> line 1059, in _build
>>     set_leaf(current_node, current_node.parent_grid,
>> current_node.l_corner, current_node.r_corner)
>>   File
>> "/u/jhwise/local/lib/python2.7/site-packages/yt-2.4-py2.7-linux-x86_64.egg/yt/utilities/amr_kdtree/amr_kdtree.py",
>> line 154, in set_leaf
>>     thisnode.grid = grid_id.id
>> AttributeError: 'NoneType' object has no attribute 'id'
>>
>>
>> I am not sure what is causing this.
>>
>> Thanks very much,
>> Renyue
>>
>>
>> On Jan 8, 2013, at 12:08 PM, Renyue Cen wrote:
>>
>> Hi Sam,
>>
>> your transpose fix DOES change things and it indeed may working now.
>> Here is the first images I got for comparison.
>> I am going to make a few more pictures to see if things are all good.
>> Thanks very much,
>> <test1.png><test2.png>
>> Best,
>> Renyue
>>
>> On Jan 8, 2013, at 11:25 AM, Sam Skillman wrote:
>>
>> Hi Renyue,
>>
>> I have actually seen this on one other system but I could never reproduce
>> it. Could you tell me the version of yt you are using? If you run "yt
>> instinfo" it should tell you. I haven't seen this bug for the last few
>> months so I thought it was gone.
>>
>> Additionally, can you try the following at the end of your script:
>>
>> im=cam.snapshot('test1.png')
>>
>> write_bitmap(im, 'test2.png', transpose=False)
>>
>> I think there may have been something wrong with the transposing of the
>> image for a short time period a few months ago.  I have very sporadic and
>> unreliable internet at AAS but I'll try to respond as quickly as possible.
>>
>> Sorry for the trouble,
>>
>> Sam
>>
>>
>> On Jan 7, 2013 10:11 AM, "Renyue Cen" <cen at astro.princeton.edu> wrote:
>>>
>>> Hi,
>>>
>>> I did some simple volume rendering with the following script:
>>>
>>>     volume2 = AMRKDTree(pf, fields=["Dark_Matter_Density"],
>>>                 no_ghost=False, tree_type="domain",
>>>                 le=c-0.5*WW, re=c+0.5*WW)
>>>     cam = pf.h.camera(c, L, W, N, tf, volume=volume2, no_ghost=False,
>>>                 north_vector=L, steady_north=True)
>>>     cam.snapshot(fn="%s_iso-DMdensity-%3.3d.png" % (filenameTHIS, j))
>>>
>>>
>>> I got rather strange results in that the pictures look symmetric, which I
>>> am pretty
>>> sure can not be true.
>>> I attach the obtained plot.
>>> Note that I am using KD tree and using 32 cores.
>>>
>>> Your help at your earliest convenience is appreciated.
>>> Best,
>>> Renyue
>>>
>>>
>>> <C15z1600.0043_iso-DMdensity-000.png>
>>
>>
>>
>>
>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>