<div dir="ltr">Hi Alex,<div><br></div><div>I think the issue is that you are using yt-3.0 to do the clump finding. I *believe* (Matt will correct me if I'm wrong) that clump finding does not work correctly in 3.0.</div>
<div><br></div><div>Can you try again in 2.6dev?</div><div><br></div><div>-Nathan</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Nov 7, 2013 at 1:07 PM, Alex Bogert <span dir="ltr"><<a href="mailto:bogart.alex@gmail.com" target="_blank">bogart.alex@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5"><div dir="ltr"><br><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div><div>Hi Nathan & Matt,<br><br>
</div>I am interested in using YT to find clumps in the Bolshoi grid particle data. So, I tried to test it with Nathan's ENZO data he gave me (DD0025). I ran into an errror.<br>
<br>Thanks for your help!<br><br></div>Alex<br><div> <br>Error:<div><div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<pre><div><div><span># set up our namespace</span>
<span>from</span> <span>yt.mods</span> <span>import</span> <span>*</span>
<span>from</span> <span>yt.analysis_modules.level_sets.api</span> <span>import</span> <span>*</span>
<span>fn</span> <span>=</span> <span>"IsolatedGalaxy/galaxy0030/galaxy0030"</span> <span># parameter file to load</span>
<span>field</span> <span>=</span> <span>"Density"</span> <span># this is the field we look for contours over -- we could do</span>
<span># this over anything. Other common choices are 'AveragedDensity'</span>
<span># and 'Dark_Matter_Density'.</span>
<span>step</span> <span>=</span> <span>2.0</span> <span># This is the multiplicative interval between contours.</span>
<span>pf</span> <span>=</span> <span>load</span><span>(</span><span>fn</span><span>)</span> <span># load data</span>
<span># We want to find clumps over the entire dataset, so we'll just grab the whole</span>
<span># thing! This is a convenience parameter that prepares an object that covers</span>
<span># the whole domain. Note, though, that it will load on demand and not before!</span>
<span>data_source</span> <span>=</span> <span>pf</span><span>.</span><span>h</span><span>.</span><span>disk</span><span>([</span><span>0.5</span><span>,</span> <span>0.5</span><span>,</span> <span>0.5</span><span>],</span> <span>[</span><span>0.</span><span>,</span> <span>0.</span><span>,</span> <span>1.</span><span>],</span>
<span>8.</span><span>/</span><span>pf</span><span>.</span><span>units</span><span>[</span><span>'kpc'</span><span>],</span> <span>1.</span><span>/</span><span>pf</span><span>.</span><span>units</span><span>[</span><span>'kpc'</span><span>])</span>
<span># Now we set some sane min/max values between which we want to find contours.</span>
<span># This is how we tell the clump finder what to look for -- it won't look for</span>
<span># contours connected below or above these threshold values.</span>
<span>c_min</span> <span>=</span> <span>10</span><span>**</span><span>na</span><span>.</span><span>floor</span><span>(</span><span>na</span><span>.</span><span>log10</span><span>(</span><span>data_source</span><span>[</span><span>field</span><span>])</span><span>.</span><span>min</span><span>()</span> <span>)</span>
<span>c_max</span> <span>=</span> <span>10</span><span>**</span><span>na</span><span>.</span><span>floor</span><span>(</span><span>na</span><span>.</span><span>log10</span><span>(</span><span>data_source</span><span>[</span><span>field</span><span>])</span><span>.</span><span>max</span><span>()</span><span>+</span><span>1</span><span>)</span>
<span># keep only clumps with at least 20 cells</span>
<span>function</span> <span>=</span> <span>'self.data[</span><span>\'</span><span>%s</span><span>\'</span><span>].size > 20'</span> <span>%</span> <span>field</span>
<span># Now find get our 'base' clump -- this one just covers the whole domain.</span>
<span>master_clump</span> <span>=</span> <span>Clump</span><span>(</span><span>data_source</span><span>,</span> <span>None</span><span>,</span> <span>field</span><span>,</span> <span>function</span><span>=</span><span>function</span><span>)</span>
<span># This next command accepts our base clump and we say the range between which</span>
<span># we want to contour. It recursively finds clumps within the master clump, at</span>
<span># intervals defined by the step size we feed it. The current value is</span>
<span># *multiplied* by step size, rather than added to it -- so this means if you</span>
<span># want to look in log10 space intervals, you would supply step = 10.0.</span>
<span>find_clumps</span><span>(</span><span>master_clump</span><span>,</span> <span>c_min</span><span>,</span> <span>c_max</span><span>,</span> <span>step</span><span>)</span>
<span># As it goes, it appends the information about all the sub-clumps to the</span>
<span># master-clump. Among different ways we can examine it, there's a convenience</span>
<span># function for outputting the full hierarchy to a file.</span>
<span>f</span> <span>=</span> <span>open</span><span>(</span><span>'</span><span>%s</span><span>_clump_hierarchy.txt'</span> <span>%</span> <span>pf</span><span>,</span><span>'w'</span><span>)</span>
<span>amods</span><span>.</span><span>level_sets</span><span>.</span><span>write_clump_hierarchy</span><span>(</span><span>master_clump</span><span>,</span><span>0</span><span>,</span><span>f</span><span>)</span>
<span>f</span><span>.</span><span>close</span><span>()</span>
<span># We can also output some handy information, as well.</span>
<span>f</span> <span>=</span> <span>open</span><span>(</span><span>'</span><span>%s</span><span>_clumps.txt'</span> <span>%</span> <span>pf</span><span>,</span><span>'w'</span><span>)</span>
<span>amods</span><span>.</span><span>level_sets</span><span>.</span><span>write_clumps</span><span>(</span><span>master_clump</span><span>,</span><span>0</span><span>,</span><span>f</span><span>)</span>
<span>f</span><span>.</span><span>close</span><span>()</span>
<span># We can traverse the clump hierarchy to get a list of all of the 'leaf' clumps</span>
<span>leaf_clumps</span> <span>=</span> <span>get_lowest_clumps</span><span>(</span><span>master_clump</span><span>)</span>
<span># If you'd like to visualize these clumps, a list of clumps can be supplied to</span>
<span># the "clumps" callback on a plot. First, we create a projection plot:</span>
<span>prj</span> <span>=</span> <span>ProjectionPlot</span><span>(</span><span>pf</span><span>,</span> <span>2</span><span>,</span> <span>field</span><span>,</span> <span>center</span><span>=</span><span>'c'</span><span>,</span> <span>width</span><span>=</span><span>(</span><span>20</span><span>,</span><span>'kpc'</span><span>))</span>
<span># Next we annotate the plot with contours on the borders of the clumps</span>
<span>prj</span><span>.</span><span>annotate_clumps</span><span>(</span><span>leaf_clumps</span><span>)</span>
<span># Lastly, we write the plot to disk.</span>
<span>prj</span><span>.</span><span>save</span><span>(</span><span>'clumps'</span><span>)</span>
<span># We can also save the clump object to disk to read in later so we don't have</span>
<span># to spend a lot of time regenerating the clump objects.</span>
<span>pf</span><span>.</span><span>h</span><span>.</span><span>save_object</span><span>(</span><span>master_clump</span><span>,</span> <span>'My_clumps'</span><span>)</span>
<span># Later, we can read in the clump object like so,</span>
<span>master_clump</span> <span>=</span> <span>pf</span><span>.</span><span>h</span><span>.</span><span>load_object</span><span>(</span><span>'My_clumps'</span></div></div><span>)<br><br><br>--------------------------------------------------------------------------------------------------------------------------------<br>
</span></pre><div><div><div><pre>yt : [INFO ] 2013-11-07 12:50:45,658 Parameters: current_time = 0.02500008999
yt : [INFO ] 2013-11-07 12:50:45,659 Parameters: domain_dimensions = [64 64 64]
yt : [INFO ] 2013-11-07 12:50:45,660 Parameters: domain_left_edge = [ 0. 0. 0.]
yt : [INFO ] 2013-11-07 12:50:45,660 Parameters: domain_right_edge = [ 1. 1. 1.]
yt : [INFO ] 2013-11-07 12:50:45,661 Parameters: cosmological_simulation = 0.0
</pre></div></div><div><div><pre><span>---------------------------------------------------------------------------</span>
<span>TypeError</span> Traceback (most recent call last)
<span><ipython-input-10-f290b737b0f4></span> in <span><module><span>()</span>
<span> 34</span> <span># *multiplied* by step size, rather than added to it -- so this means if you</span><span></span><span></span></span>
<span> 35</span> <span># want to look in log10 space intervals, you would supply step = 10.0.</span><span></span><span></span>
<span>---> 36<span> </span>find_clumps</span><span>(</span>master_clump<span>,</span> c_min<span>,</span> c_max<span>,</span> step<span>)</span><span></span>
<span> 37</span> <span></span>
<span> 38</span> <span># As it goes, it appends the information about all the sub-clumps to the</span><span></span><span></span>
<span>/usr/local/yt-conda/src/yt-hg/yt/analysis_modules/level_sets/clump_handling.py</span> in <span>find_clumps<span>(clump, min_val, max_val, d_clump)</span>
<span> 168</span> <span>print</span> <span>"Finding clumps: min: %e, max: %e, step: %f"</span> <span>%</span> <span>(</span></span>min_val<span>,</span> max_val<span>,</span> d_clump<span>)</span><span></span>
<span> 169</span> <span>if</span> min_val <span>>=</span> max_val<span>:</span> <span>return</span><span></span>
<span>--> 170<span> </span>clump</span><span>.</span>find_children<span>(</span>min_val<span>)</span><span></span>
<span> 171</span> <span></span>
<span> 172</span> <span>if</span> <span>(</span>len<span>(</span>clump<span>.</span>children<span>)</span> <span>==</span> <span>1</span><span>)</span><span>:</span><span></span>
<span>/usr/local/yt-conda/src/yt-hg/yt/analysis_modules/level_sets/clump_handling.py</span> in <span>find_children<span>(self, min_val, max_val)</span>
<span> 109</span> <span>if</span> </span>max_val <span>is</span> None<span>:</span> max_val <span>=</span> self<span>.</span>max_val<span></span>
<span> 110</span> contour_info = identify_contours(self.data, self.field, min_val, max_val,
<span>--> 111<span> self.cached_fields)
</span><span> 112</span> <span>for</span> </span>cid <span>in</span> contour_info<span>:</span><span></span>
<span> 113</span> new_clump <span>=</span> self<span>.</span>data<span>.</span>extract_region<span>(</span>contour_info<span>[</span>cid<span>]</span><span>)</span><span></span>
<span>/usr/local/yt-conda/src/yt-hg/yt/analysis_modules/level_sets/contour_finder.py</span> in <span>identify_contours<span>(data_source, field, min_val, max_val, cached_fields)</span>
<span> 59</span> def identify_contours(data_source, field, min_val, max_val,
<span> 60</span> cached_fields=None):
<span>---> 61<span> </span>cur_max_id</span> <span>=</span> </span>np<span>.</span>sum<span>(</span><span>[</span>g<span>.</span>ActiveDimensions<span>.</span>prod<span>(</span><span>)</span> <span>for</span> g <span>in</span> data_source<span>.</span>_grids<span>]</span><span>)</span><span></span>
<span> 62</span> pbar <span>=</span> get_pbar<span>(</span><span>"First pass"</span><span>,</span> len<span>(</span>data_source<span>.</span>_grids<span>)</span><span>)</span><span></span>
<span> 63</span> grids <span>=</span> sorted<span>(</span>data_source<span>.</span>_grids<span>,</span> key<span>=</span><span>lambda</span> g<span>:</span> <span>-</span>g<span>.</span>Level<span>)</span><span></span>
<span>TypeError</span>: 'NoneType' object is not iterable
</pre></div></div><div><div><pre>Finding clumps: min: 1.000000e-30, max: 1.000000e-21, step: 2.000000
</pre></div></div></div><pre><br></pre></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Nov 7, 2013 at 12:49 PM, Alex Bogert <span dir="ltr"><<a href="mailto:bogart.alex@gmail.com" target="_blank">bogart.alex@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><br><div class="gmail_quote">---------- Forwarded message ----------<br>From: <b class="gmail_sendername">Alex Bogert</b> <span dir="ltr"><<a href="mailto:bogart.alex@gmail.com" target="_blank">bogart.alex@gmail.com</a>></span><br>
Date: Thu, Nov 7, 2013 at 12:46 PM<br>Subject: yt - clump finding<br>To: "Conor Kaminer (Google Drive)" <<a href="mailto:ckaminer@ucsc.edu" target="_blank">ckaminer@ucsc.edu</a>><br><br><br><div dir="ltr">
<pre><span># set up our namespace</span>
<span>from</span> <span>yt.mods</span> <span>import</span> <span>*</span>
<span>from</span> <span>yt.analysis_modules.level_sets.api</span> <span>import</span> <span>*</span>
<span>fn</span> <span>=</span> <span>"IsolatedGalaxy/galaxy0030/galaxy0030"</span> <span># parameter file to load</span>
<span>field</span> <span>=</span> <span>"Density"</span> <span># this is the field we look for contours over -- we could do</span>
<span># this over anything. Other common choices are 'AveragedDensity'</span>
<span># and 'Dark_Matter_Density'.</span>
<span>step</span> <span>=</span> <span>2.0</span> <span># This is the multiplicative interval between contours.</span>
<span>pf</span> <span>=</span> <span>load</span><span>(</span><span>fn</span><span>)</span> <span># load data</span>
<span># We want to find clumps over the entire dataset, so we'll just grab the whole</span>
<span># thing! This is a convenience parameter that prepares an object that covers</span>
<span># the whole domain. Note, though, that it will load on demand and not before!</span>
<span>data_source</span> <span>=</span> <span>pf</span><span>.</span><span>h</span><span>.</span><span>disk</span><span>([</span><span>0.5</span><span>,</span> <span>0.5</span><span>,</span> <span>0.5</span><span>],</span> <span>[</span><span>0.</span><span>,</span> <span>0.</span><span>,</span> <span>1.</span><span>],</span>
<span>8.</span><span>/</span><span>pf</span><span>.</span><span>units</span><span>[</span><span>'kpc'</span><span>],</span> <span>1.</span><span>/</span><span>pf</span><span>.</span><span>units</span><span>[</span><span>'kpc'</span><span>])</span>
<span># Now we set some sane min/max values between which we want to find contours.</span>
<span># This is how we tell the clump finder what to look for -- it won't look for</span>
<span># contours connected below or above these threshold values.</span>
<span>c_min</span> <span>=</span> <span>10</span><span>**</span><span>na</span><span>.</span><span>floor</span><span>(</span><span>na</span><span>.</span><span>log10</span><span>(</span><span>data_source</span><span>[</span><span>field</span><span>])</span><span>.</span><span>min</span><span>()</span> <span>)</span>
<span>c_max</span> <span>=</span> <span>10</span><span>**</span><span>na</span><span>.</span><span>floor</span><span>(</span><span>na</span><span>.</span><span>log10</span><span>(</span><span>data_source</span><span>[</span><span>field</span><span>])</span><span>.</span><span>max</span><span>()</span><span>+</span><span>1</span><span>)</span>
<span># keep only clumps with at least 20 cells</span>
<span>function</span> <span>=</span> <span>'self.data[</span><span>\'</span><span>%s</span><span>\'</span><span>].size > 20'</span> <span>%</span> <span>field</span>
<span># Now find get our 'base' clump -- this one just covers the whole domain.</span>
<span>master_clump</span> <span>=</span> <span>Clump</span><span>(</span><span>data_source</span><span>,</span> <span>None</span><span>,</span> <span>field</span><span>,</span> <span>function</span><span>=</span><span>function</span><span>)</span>
<span># This next command accepts our base clump and we say the range between which</span>
<span># we want to contour. It recursively finds clumps within the master clump, at</span>
<span># intervals defined by the step size we feed it. The current value is</span>
<span># *multiplied* by step size, rather than added to it -- so this means if you</span>
<span># want to look in log10 space intervals, you would supply step = 10.0.</span>
<span>find_clumps</span><span>(</span><span>master_clump</span><span>,</span> <span>c_min</span><span>,</span> <span>c_max</span><span>,</span> <span>step</span><span>)</span>
<span># As it goes, it appends the information about all the sub-clumps to the</span>
<span># master-clump. Among different ways we can examine it, there's a convenience</span>
<span># function for outputting the full hierarchy to a file.</span>
<span>f</span> <span>=</span> <span>open</span><span>(</span><span>'</span><span>%s</span><span>_clump_hierarchy.txt'</span> <span>%</span> <span>pf</span><span>,</span><span>'w'</span><span>)</span>
<span>amods</span><span>.</span><span>level_sets</span><span>.</span><span>write_clump_hierarchy</span><span>(</span><span>master_clump</span><span>,</span><span>0</span><span>,</span><span>f</span><span>)</span>
<span>f</span><span>.</span><span>close</span><span>()</span>
<span># We can also output some handy information, as well.</span>
<span>f</span> <span>=</span> <span>open</span><span>(</span><span>'</span><span>%s</span><span>_clumps.txt'</span> <span>%</span> <span>pf</span><span>,</span><span>'w'</span><span>)</span>
<span>amods</span><span>.</span><span>level_sets</span><span>.</span><span>write_clumps</span><span>(</span><span>master_clump</span><span>,</span><span>0</span><span>,</span><span>f</span><span>)</span>
<span>f</span><span>.</span><span>close</span><span>()</span>
<span># We can traverse the clump hierarchy to get a list of all of the 'leaf' clumps</span>
<span>leaf_clumps</span> <span>=</span> <span>get_lowest_clumps</span><span>(</span><span>master_clump</span><span>)</span>
<span># If you'd like to visualize these clumps, a list of clumps can be supplied to</span>
<span># the "clumps" callback on a plot. First, we create a projection plot:</span>
<span>prj</span> <span>=</span> <span>ProjectionPlot</span><span>(</span><span>pf</span><span>,</span> <span>2</span><span>,</span> <span>field</span><span>,</span> <span>center</span><span>=</span><span>'c'</span><span>,</span> <span>width</span><span>=</span><span>(</span><span>20</span><span>,</span><span>'kpc'</span><span>))</span>
<span># Next we annotate the plot with contours on the borders of the clumps</span>
<span>prj</span><span>.</span><span>annotate_clumps</span><span>(</span><span>leaf_clumps</span><span>)</span>
<span># Lastly, we write the plot to disk.</span>
<span>prj</span><span>.</span><span>save</span><span>(</span><span>'clumps'</span><span>)</span>
<span># We can also save the clump object to disk to read in later so we don't have</span>
<span># to spend a lot of time regenerating the clump objects.</span>
<span>pf</span><span>.</span><span>h</span><span>.</span><span>save_object</span><span>(</span><span>master_clump</span><span>,</span> <span>'My_clumps'</span><span>)</span>
<span># Later, we can read in the clump object like so,</span>
<span>master_clump</span> <span>=</span> <span>pf</span><span>.</span><span>h</span><span>.</span><span>load_object</span><span>(</span><span>'My_clumps'</span><span>)</span></pre>
</div>
</div><br></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div></div></div></div></div>
</blockquote></div><br></div></div>
</div></div><br>_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
<br></blockquote></div><br></div>