[yt-users] yt - clump finding

[Matthew Turk]

Hi Alex and Nathan,

That is indeed correct.  I have a pull request to change this:

https://bitbucket.org/yt_analysis/yt-3.0/pull-request/120/contour-finder-rewrite

but it's not on the roadmap for the next week or so.

-Matt

On Thu, Nov 7, 2013 at 6:53 PM, Nathan Goldbaum <nathan12343 at gmail.com> wrote:
> Hi Alex,
>
> I think the issue is that you are using yt-3.0 to do the clump finding.  I
> *believe* (Matt will correct me if I'm wrong) that clump finding does not
> work correctly in 3.0.
>
> Can you try again in 2.6dev?
>
> -Nathan
>
>
> On Thu, Nov 7, 2013 at 1:07 PM, Alex Bogert <bogart.alex at gmail.com> wrote:
>>
>>
>>> Hi Nathan & Matt,
>>>
>>> I am interested in using YT to find clumps in the Bolshoi grid particle
>>> data. So, I tried to test it with Nathan's ENZO data he gave me (DD0025). I
>>> ran into an errror.
>>>
>>> Thanks for your help!
>>>
>>> Alex
>>>
>>> Error:
>>>>
>>>> # set up our namespace
>>>> from yt.mods import *
>>>> from yt.analysis_modules.level_sets.api import *
>>>>
>>>> fn = "IsolatedGalaxy/galaxy0030/galaxy0030" # parameter file to load
>>>> field = "Density" # this is the field we look for contours over -- we
>>>> could do
>>>>                   # this over anything.  Other common choices are
>>>> 'AveragedDensity'
>>>>                   # and 'Dark_Matter_Density'.
>>>> step = 2.0 # This is the multiplicative interval between contours.
>>>>
>>>> pf = load(fn) # load data
>>>>
>>>> # We want to find clumps over the entire dataset, so we'll just grab the
>>>> whole
>>>> # thing!  This is a convenience parameter that prepares an object that
>>>> covers
>>>> # the whole domain.  Note, though, that it will load on demand and not
>>>> before!
>>>> data_source = pf.h.disk([0.5, 0.5, 0.5], [0., 0., 1.],
>>>>                         8./pf.units['kpc'], 1./pf.units['kpc'])
>>>>
>>>> # Now we set some sane min/max values between which we want to find
>>>> contours.
>>>> # This is how we tell the clump finder what to look for -- it won't look
>>>> for
>>>> # contours connected below or above these threshold values.
>>>> c_min = 10**na.floor(na.log10(data_source[field]).min()  )
>>>> c_max = 10**na.floor(na.log10(data_source[field]).max()+1)
>>>>
>>>> # keep only clumps with at least 20 cells
>>>> function = 'self.data[\'%s\'].size > 20' % field
>>>>
>>>> # Now find get our 'base' clump -- this one just covers the whole
>>>> domain.
>>>> master_clump = Clump(data_source, None, field, function=function)
>>>>
>>>> # This next command accepts our base clump and we say the range between
>>>> which
>>>> # we want to contour.  It recursively finds clumps within the master
>>>> clump, at
>>>> # intervals defined by the step size we feed it.  The current value is
>>>> # *multiplied* by step size, rather than added to it -- so this means if
>>>> you
>>>> # want to look in log10 space intervals, you would supply step = 10.0.
>>>> find_clumps(master_clump, c_min, c_max, step)
>>>>
>>>> # As it goes, it appends the information about all the sub-clumps to the
>>>> # master-clump.  Among different ways we can examine it, there's a
>>>> convenience
>>>> # function for outputting the full hierarchy to a file.
>>>> f = open('%s_clump_hierarchy.txt' % pf,'w')
>>>> amods.level_sets.write_clump_hierarchy(master_clump,0,f)
>>>> f.close()
>>>>
>>>> # We can also output some handy information, as well.
>>>> f = open('%s_clumps.txt' % pf,'w')
>>>> amods.level_sets.write_clumps(master_clump,0,f)
>>>> f.close()
>>>>
>>>> # We can traverse the clump hierarchy to get a list of all of the 'leaf'
>>>> clumps
>>>> leaf_clumps = get_lowest_clumps(master_clump)
>>>>
>>>> # If you'd like to visualize these clumps, a list of clumps can be
>>>> supplied to
>>>> # the "clumps" callback on a plot.  First, we create a projection plot:
>>>> prj = ProjectionPlot(pf, 2, field, center='c', width=(20,'kpc'))
>>>>
>>>> # Next we annotate the plot with contours on the borders of the clumps
>>>> prj.annotate_clumps(leaf_clumps)
>>>>
>>>> # Lastly, we write the plot to disk.
>>>> prj.save('clumps')
>>>>
>>>> # We can also save the clump object to disk to read in later so we don't
>>>> have
>>>> # to spend a lot of time regenerating the clump objects.
>>>> pf.h.save_object(master_clump, 'My_clumps')
>>>>
>>>> # Later, we can read in the clump object like so,
>>>> master_clump = pf.h.load_object('My_clumps'
>>>> )
>>>>
>>>>
>>>>
>>>> --------------------------------------------------------------------------------------------------------------------------------
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> yt : [INFO     ] 2013-11-07 12:50:45,658 Parameters: current_time
>>>> = 0.02500008999
>>>> yt : [INFO     ] 2013-11-07 12:50:45,659 Parameters: domain_dimensions
>>>> = [64 64 64]
>>>> yt : [INFO     ] 2013-11-07 12:50:45,660 Parameters: domain_left_edge
>>>> = [ 0.  0.  0.]
>>>> yt : [INFO     ] 2013-11-07 12:50:45,660 Parameters: domain_right_edge
>>>> = [ 1.  1.  1.]
>>>> yt : [INFO     ] 2013-11-07 12:50:45,661 Parameters:
>>>> cosmological_simulation   = 0.0
>>>>
>>>>
>>>> ---------------------------------------------------------------------------
>>>> TypeError                                 Traceback (most recent call
>>>> last)
>>>> <ipython-input-10-f290b737b0f4> in <module>()
>>>>      34 # *multiplied* by step size, rather than added to it -- so this
>>>> means if you
>>>>      35 # want to look in log10 space intervals, you would supply step =
>>>> 10.0.
>>>> ---> 36 find_clumps(master_clump, c_min, c_max, step)
>>>>      37
>>>>      38 # As it goes, it appends the information about all the
>>>> sub-clumps to the
>>>>
>>>>
>>>> /usr/local/yt-conda/src/yt-hg/yt/analysis_modules/level_sets/clump_handling.py
>>>> in find_clumps(clump, min_val, max_val, d_clump)
>>>>     168     print "Finding clumps: min: %e, max: %e, step: %f" %
>>>> (min_val, max_val, d_clump)
>>>>     169     if min_val >= max_val: return
>>>> --> 170     clump.find_children(min_val)
>>>>     171
>>>>     172     if (len(clump.children) == 1):
>>>>
>>>>
>>>> /usr/local/yt-conda/src/yt-hg/yt/analysis_modules/level_sets/clump_handling.py
>>>> in find_children(self, min_val, max_val)
>>>>     109         if max_val is None: max_val = self.max_val
>>>>     110         contour_info = identify_contours(self.data, self.field,
>>>> min_val, max_val,
>>>> --> 111                                          self.cached_fields)
>>>>     112         for cid in contour_info:
>>>>     113             new_clump =
>>>> self.data.extract_region(contour_info[cid])
>>>>
>>>>
>>>> /usr/local/yt-conda/src/yt-hg/yt/analysis_modules/level_sets/contour_finder.py
>>>> in identify_contours(data_source, field, min_val, max_val, cached_fields)
>>>>      59 def identify_contours(data_source, field, min_val, max_val,
>>>>      60                           cached_fields=None):
>>>> ---> 61     cur_max_id = np.sum([g.ActiveDimensions.prod() for g in
>>>> data_source._grids])
>>>>      62     pbar = get_pbar("First pass", len(data_source._grids))
>>>>      63     grids = sorted(data_source._grids, key=lambda g: -g.Level)
>>>>
>>>> TypeError: 'NoneType' object is not iterable
>>>>
>>>> Finding clumps: min: 1.000000e-30, max: 1.000000e-21, step: 2.000000
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Nov 7, 2013 at 12:49 PM, Alex Bogert <bogart.alex at gmail.com>
>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>> ---------- Forwarded message ----------
>>>>> From: Alex Bogert <bogart.alex at gmail.com>
>>>>> Date: Thu, Nov 7, 2013 at 12:46 PM
>>>>> Subject: yt - clump finding
>>>>> To: "Conor Kaminer (Google Drive)" <ckaminer at ucsc.edu>
>>>>>
>>>>>
>>>>> # set up our namespace
>>>>> from yt.mods import *
>>>>> from yt.analysis_modules.level_sets.api import *
>>>>>
>>>>> fn = "IsolatedGalaxy/galaxy0030/galaxy0030" # parameter file to load
>>>>> field = "Density" # this is the field we look for contours over -- we
>>>>> could do
>>>>>                   # this over anything.  Other common choices are
>>>>> 'AveragedDensity'
>>>>>                   # and 'Dark_Matter_Density'.
>>>>> step = 2.0 # This is the multiplicative interval between contours.
>>>>>
>>>>> pf = load(fn) # load data
>>>>>
>>>>> # We want to find clumps over the entire dataset, so we'll just grab
>>>>> the whole
>>>>> # thing!  This is a convenience parameter that prepares an object that
>>>>> covers
>>>>> # the whole domain.  Note, though, that it will load on demand and not
>>>>> before!
>>>>> data_source = pf.h.disk([0.5, 0.5, 0.5], [0., 0., 1.],
>>>>>                         8./pf.units['kpc'], 1./pf.units['kpc'])
>>>>>
>>>>> # Now we set some sane min/max values between which we want to find
>>>>> contours.
>>>>> # This is how we tell the clump finder what to look for -- it won't
>>>>> look for
>>>>> # contours connected below or above these threshold values.
>>>>> c_min = 10**na.floor(na.log10(data_source[field]).min()  )
>>>>> c_max = 10**na.floor(na.log10(data_source[field]).max()+1)
>>>>>
>>>>> # keep only clumps with at least 20 cells
>>>>> function = 'self.data[\'%s\'].size > 20' % field
>>>>>
>>>>> # Now find get our 'base' clump -- this one just covers the whole
>>>>> domain.
>>>>> master_clump = Clump(data_source, None, field, function=function)
>>>>>
>>>>> # This next command accepts our base clump and we say the range between
>>>>> which
>>>>> # we want to contour.  It recursively finds clumps within the master
>>>>> clump, at
>>>>> # intervals defined by the step size we feed it.  The current value is
>>>>> # *multiplied* by step size, rather than added to it -- so this means
>>>>> if you
>>>>> # want to look in log10 space intervals, you would supply step = 10.0.
>>>>> find_clumps(master_clump, c_min, c_max, step)
>>>>>
>>>>> # As it goes, it appends the information about all the sub-clumps to
>>>>> the
>>>>> # master-clump.  Among different ways we can examine it, there's a
>>>>> convenience
>>>>> # function for outputting the full hierarchy to a file.
>>>>> f = open('%s_clump_hierarchy.txt' % pf,'w')
>>>>> amods.level_sets.write_clump_hierarchy(master_clump,0,f)
>>>>> f.close()
>>>>>
>>>>> # We can also output some handy information, as well.
>>>>> f = open('%s_clumps.txt' % pf,'w')
>>>>> amods.level_sets.write_clumps(master_clump,0,f)
>>>>> f.close()
>>>>>
>>>>> # We can traverse the clump hierarchy to get a list of all of the
>>>>> 'leaf' clumps
>>>>> leaf_clumps = get_lowest_clumps(master_clump)
>>>>>
>>>>> # If you'd like to visualize these clumps, a list of clumps can be
>>>>> supplied to
>>>>> # the "clumps" callback on a plot.  First, we create a projection plot:
>>>>> prj = ProjectionPlot(pf, 2, field, center='c', width=(20,'kpc'))
>>>>>
>>>>> # Next we annotate the plot with contours on the borders of the clumps
>>>>> prj.annotate_clumps(leaf_clumps)
>>>>>
>>>>> # Lastly, we write the plot to disk.
>>>>> prj.save('clumps')
>>>>>
>>>>> # We can also save the clump object to disk to read in later so we
>>>>> don't have
>>>>> # to spend a lot of time regenerating the clump objects.
>>>>> pf.h.save_object(master_clump, 'My_clumps')
>>>>>
>>>>> # Later, we can read in the clump object like so,
>>>>> master_clump = pf.h.load_object('My_clumps')
>>>>>
>>>>>
>>>>
>>>
>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>