[yt-users] Athena phase plot problem
Stephanie Tonnesen
stonnes at gmail.com
Tue Dec 18 13:27:46 PST 2012
Hi John,
The weighted average quantity call of specific_scalar[0] worked.
The domain info:
pf.domain_right_edge
array([ 3.2, 3.2, 3.8])
>>> pf.domain_dimensions
array([1048, 1048, 950])
And then using id0 and id1:
pf2.domain_right_edge
array([-1.349618, -2.259542, -1.145263])
>>> pf2.domain_dimensions
array([303, 154, 140])
When I combined the vtk files into a single 25 GB file, I get an error when
creating the PlotCollection:
pc = PlotCollection(pf)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File
"/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/visualization/plot_collection.py",
line 121, in __init__
v,self.c = pf.h.find_max("Density") # @todo: ensure no caching
File
"/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/object_finding_mixin.py",
line 61, in find_max
mg, mc, mv, pos = self.find_max_cell_location(field, finest_levels)
File
"/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/object_finding_mixin.py",
line 74, in find_max_cell_location
source.quantities["MaxLocation"]( field, lazy_reader=True)
File
"/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py",
line 92, in __call__
return self._call_func_lazy(args, kwargs)
File
"/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py",
line 99, in _call_func_lazy
rv = self.func(GridChildMaskWrapper(g, self._data_source), *args,
**kwargs)
File
"/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py",
line 649, in _MaxLocation
mx, my, mz = [data[ax][maxi] for ax in 'xyz']
File
"/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/derived_quantities.py",
line 60, in __getitem__
data = self.data_source._get_data_from_grid(self.grid, item)
File
"/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py",
line 93, in save_state
tr = func(self, grid, field, *args, **kwargs)
File
"/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/data_containers.py",
line 2630, in _get_data_from_grid
tr = grid[field]
File
"/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.py",
line 157, in __getitem__
self.get_data(key)
File
"/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.py",
line 200, in get_data
self._generate_field(field)
File
"/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/grid_patch.py",
line 145, in _generate_field
self[field] = self.pf.field_info[field](self)
File
"/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/field_info_container.py",
line 378, in __call__
dd = self._function(self, data)
File
"/u/stonnes/Installs/yt-x86_64/src/yt-hg/yt/data_objects/universal_fields.py",
line 94, in _coordX
return (np.ones(data.ActiveDimensions, dtype='float64')
File
"/u/stonnes/Installs/yt-x86_64/lib/python2.7/site-packages/numpy/core/numeric.py",
line 1819, in ones
a = empty(shape, dtype, order)
MemoryError
On Tue, Dec 18, 2012 at 1:49 PM, John ZuHone <jzuhone at gmail.com> wrote:
> Hi Stephanie,
>
> I doubt that 25 GB is too big.
>
> Can you also try a weighted average quantity of specific_scalar[0] itself?
>
> When you use only id0 and id1, you are potentially also changing
> domain_right_edge, since for Athena yt has to go through all of the id*/*
> to find the largest right edge for any grid to determine domain_right_edge.
>
> Can you figure out what pf.domain_right_edge and pf.domain_dimensions are
> for the two different cases (where you have all the data and also where you
> have just id0 and id1)?
>
> I'm not sure at the moment what that would tell us, but I'm curious about
> it.
>
> As another possible way to get some clues, I am aware that Athena has a
> tool that joins the vtk files into one. Is it possible to run that and then
> try the same script on the merged file?
>
> I'm also wondering if Matt and Sam have any thoughts on this.
>
> Best,
>
> John
>
> On Dec 18, 2012, at 1:21 PM, Stephanie Tonnesen <stonnes at gmail.com> wrote:
>
> Hi John,
>
> So, what I mean by "expected" variables are basically all variables BUT
> specific_scalar[0]. When I read in the vtk file, you can see that the
> bottom line says
>
> yt : [INFO ] 2012-12-13 23:27:13,214 Adding unknown field specific_scalar[0] to list of fields
>
>
> Yes, I can do any of the "quantities" commands, for example:
>
> alld = pf.h.all_data()
> tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25",
> "grid['z'] > 0.01"])
>
> radmean = tracerp25.quantities["WeightedAverageQuantity"]("Radius","Ones")
>
> Slicing also works:
>
> p = SlicePlot(pf,"x","specific_scalar[0]")
>
> Now I am starting to worry that it is because output is so big--each vtk
> file is 72 MB, so a full output is almost 25 GB. When I just use two
> outputs (moving them so yt only sees id0 and id1 folders) the phase plot
> works. But, after running the add_phase_object command on the whole data
> set for more than 1.5 hours I still have no result and my computer keeps
> freezing--although top shows that only 42% of memory is being used.
>
> Is it possible that my output is just too big?
>
> Stephanie
>
>
> On Tue, Dec 18, 2012 at 11:00 AM, John Zuhone <jzuhone at gmail.com> wrote:
>
>> Hi Stephanie,
>>
>> Ok, thanks for checking that.
>>
>> I guess I'm not sure what you mean by the "expected" set of variables. Do
>> you mean the ones that yt defines, or the ones that are in the field list?
>> It looks like "velocity_z" works for one call and doesn't for another.
>>
>> Are you able to do anything else with "specific_scalar[0]"? Like make a
>> slice, or a projection of some sort? Or maybe take the weighted average of
>> it in a sphere. I'd be interested in seeing if the problem was restricted
>> to phase plotting.
>>
>> How big is the dataset, by the way? Is there something that fails in the
>> same way that the both of us can test on?
>>
>> Best,
>>
>> John
>>
>>
>> On Tue, Dec 18, 2012 at 10:53 AM, Stephanie Tonnesen <stonnes at gmail.com>wrote:
>>
>>>
>>> Hi,
>>>
>>> Yup. Just to be sure I updated again and reattempted to make the
>>> phase_object and python was Killed again.
>>>
>>> Thanks!
>>>
>>> Stephanie
>>>
>>>
>>>
>>> On Tue, Dec 18, 2012 at 12:35 AM, John ZuHone <jzuhone at gmail.com> wrote:
>>>
>>>> Hi Stephanie,
>>>>
>>>> Just want to make sure that you're using the latest version of the yt
>>>> development tree--it contains some important Athena fixes that were put in
>>>> a few weeks back.
>>>>
>>>> If so, then we'll work from there to figure out what's gone wrong.
>>>>
>>>> Best,
>>>>
>>>> John
>>>>
>>>> On Dec 17, 2012, at 5:44 PM, Stephanie Tonnesen <stonnes at gmail.com>
>>>> wrote:
>>>>
>>>> Hi yt-users,
>>>>
>>>> I am trying to make phase_objects of Athena data. I am able to make
>>>> phase_objects if I use the "expected" set of variables (will show below),
>>>> but kill my kernel or am kicked out of python if I try and use an "unknown
>>>> field" read in from a set of vtk files.
>>>>
>>>>
>>>>
>>>> from yt.mods import *
>>>>
>>>> pf = load("id0/rps.0080.vtk")
>>>>
>>>> yt : [INFO ] 2012-12-13 23:27:08,678 Temporarily setting domain_right_edge = -domain_left_edge. This will be corrected automatically if it is not the case.
>>>> yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: current_time = 40.00096
>>>> yt : [INFO ] 2012-12-13 23:27:08,974 Parameters: domain_dimensions = [1048 1048 655]
>>>> yt : [INFO ] 2012-12-13 23:27:08,975 Parameters: domain_left_edge = [-3.2 -3.2 -2. ]
>>>> yt : [INFO ] 2012-12-13 23:27:08,976 Parameters: domain_right_edge = [ 3.2 3.2 2. ]
>>>> yt : [INFO ] 2012-12-13 23:27:08,977 Parameters: cosmological_simulation = 0.0
>>>> yt : [INFO ] 2012-12-13 23:27:13,214 Adding unknown field specific_scalar[0] to list of fields
>>>>
>>>> pf.h.field_list
>>>>
>>>> ['density',
>>>> 'pressure',
>>>> 'velocity_z',
>>>> 'velocity_x',
>>>> 'velocity_y',
>>>> 'specific_scalar[0]']
>>>>
>>>> alld = pf.h.all_data()
>>>> tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25",
>>>> "grid['z'] > 0.01"])
>>>>
>>>> ##These commands works:
>>>>
>>>> radmean = tracerp25.quantities["WeightedAverageQuantity"]("Radius","Ones")
>>>>
>>>> pc = PlotCollectionInteractive(pf)
>>>>
>>>> pc.add_phase_object(alld,["z","velocity_z","CellMass"],weight=None)
>>>>
>>>> #The calls below fail. There is no error message. If I am in iPython,
>>>> #the kernel dies. If I am running on a computer, it freezes and
>>>> #eventually I just get the message
>>>> #Killed
>>>>
>>>> pc.add_phase_object(alld,["z","specific_scalar[0]","CellMass"],weight=None)
>>>> pc.add_phase_object(tracerp25,["z","velocity_z","CellMass"],weight=None)
>>>>
>>>> #If I read in a single vtk file ("id122/rps-id122.0080.vtk") the above
>>>> #commands work
>>>>
>>>> Any help would be much appreciated! Thanks,
>>>>
>>>> Stephanie
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>
>>
>> --
>> John ZuHone
>>
>> Postdoctoral Researcher
>> NASA/Goddard Space Flight Center
>>
>> jzuhone at gmail.com
>> john.zuhone at nasa.gov
>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20121218/dfc0c02a/attachment.html>
More information about the yt-users
mailing list