[yt-users] running out of memory in a loop

[John ZuHone]

Hi Stephanie,

The Athena frontend for yt has been developed pretty well but hasn't been optimized, so I'll have to take a look sometime soon and see where it may be hogging memory. 

Best,

John

John ZuHone
Laboratory for High-Energy Astrophysics
NASA/Goddard Space Flight Center
8800 Greenbelt Rd., Code 662
Greenbelt, MD 20771
(w) 301-286-2531
(m) 773-758-0172
jzuhone at gmail.com
john.zuhone at nasa.gov

> On Oct 23, 2013, at 9:17 PM, Stephanie Tonnesen <stonnes at gmail.com> wrote:
> 
> Thanks!  This didn't work, so I guess it must be Athena.  I would not think that is the problem because it reads in the second file, makes the SlicePlot and ProjectionPlot, and just dies with memory while making the phase_object.  Ah, well, I can work around it.  
> 
> -Stephanie
> 
> 
>> On Wed, Oct 23, 2013 at 3:53 PM, Matthew Turk <matthewturk at gmail.com> wrote:
>> Yup, that's right.
>> 
>> 
>>> On Wed, Oct 23, 2013 at 3:52 PM, Stephanie Tonnesen <stonnes at gmail.com> wrote:
>>> Okay, so to be totally clear:
>>> 
>>> I put import gc as the first line,
>>> and gc.collect() as the last line, but still indented so it is in the loop?
>>> 
>>> import gc
>>> 
>>> import matplotlib.pylab as pl
>>> from mpl_toolkits.mplot3d import Axes3D
>>> from yt.mods import *
>>> 
>>> ......
>>> 
>>> i = 0
>>> while i < len(outstring):
>>>     pf = load("id0/rps."+outstring[i]+"
>>> .vtk",parameters={"TimeUnits":3.086e14,"LengthUnits":8.0236e22,"DensityUnits":9.999e-30,"VelocityUnits":2.6e8})
>>> 
>>> 
>>>     p = SlicePlot(pf,"x","Density")
>>>     p.set_log("Density", True)
>>>     p.save()
>>>     
>>>     
>>>     p2 = ProjectionPlot(pf,"x","Density")
>>>     p2.set_log("Density", True)
>>>     p2.save()
>>>     
>>>     
>>>     alld = pf.h.all_data()
>>> 
>>>     pc = PlotCollection(pf)
>>>     tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25", "grid['z'] > 0.04"])
>>>     tp25extrema = tracerp25.quantities["Extrema"]("Density")
>>>     pc.add_phase_object(tracerp25,["zkpc","z-velocity","CellMassSolar"],weight=None,x_bins=50,y_bins=50,y_bounds = [-1e8,1.5e8],x_log=False,y_log=False)
>>>     pc.set_zlim(10000,1e10)
>>>     pc.save()
>>>     
>>>     i = i+1
>>>     gc.collect()
>>> 
>>> 
>>> 
>>> 
>>>> On Wed, Oct 23, 2013 at 3:36 PM, Matthew Turk <matthewturk at gmail.com> wrote:
>>>> Hi Stephanie,
>>>> 
>>>> Can you try adding this at the top of your script:
>>>> 
>>>> import gc
>>>> 
>>>> and just after i + i + 1 putting:
>>>> 
>>>> gc.collect()
>>>> 
>>>> If this fixes the problem, then there may just be extra references sitting around.  If not, then I suspect there may be an issue with the Athena frontend hanging onto data.
>>>> 
>>>> -Matt
>>>> 
>>>> 
>>>>> On Wed, Oct 23, 2013 at 3:20 PM, Stephanie Tonnesen <stonnes at gmail.com> wrote:
>>>>> Hi everyone,
>>>>> 
>>>>> Okay, so I am trying to make a SlicePlot, a ProjectionPlot, and a phase_object for a bunch of different outputs.  My code keeps using up all my memory and dying after the first loop.  I am not using variables, I think, so how do I free up my memory at the end of each loop?
>>>>> 
>>>>> See code below.
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Stephanie
>>>>> 
>>>>> 
>>>>> import matplotlib.pylab as pl
>>>>> from mpl_toolkits.mplot3d import Axes3D
>>>>> from yt.mods import *
>>>>> 
>>>>> def _xkpc(field,data):
>>>>>     return(data["x"]*26.)
>>>>> 
>>>>> def _ykpc(field,data):
>>>>>     return(data["y"]*26.)
>>>>> 
>>>>> def _zkpc(field,data):
>>>>>     return(data["z"]*26.)
>>>>> 
>>>>> def _CellMassSolar(field,data):
>>>>>     return(data["CellMass"]/1.99e33)
>>>>> 
>>>>> add_field("xkpc",function=_xkpc,units="kpc")
>>>>> add_field("ykpc",function=_ykpc,units="kpc")
>>>>> add_field("zkpc",function=_zkpc,units="kpc")
>>>>> add_field("CellMassSolar",function=_CellMassSolar,units=r'M$_{\odot}$')
>>>>> 
>>>>> outstring = ['0020','0030','0040','0050','0060','0070','0080','0090','0100','0110','0120','0130']
>>>>> i = 0
>>>>> while i < len(outstring):
>>>>>     pf = load("id0/rps."+outstring[i]+".vtk",parameters={"TimeUnits":3.086e14,"LengthUnits":8.0236e22,"DensityUnits":9.999e-30,"VelocityUnits":2.6e8})
>>>>> 
>>>>> 
>>>>>     p = SlicePlot(pf,"x","Density")
>>>>>     p.set_log("Density", True)
>>>>>     p.save()
>>>>>     
>>>>>     
>>>>>     p2 = ProjectionPlot(pf,"x","Density")
>>>>>     p2.set_log("Density", True)
>>>>>     p2.save()
>>>>>     
>>>>>     
>>>>>     alld = pf.h.all_data()
>>>>> 
>>>>>     pc = PlotCollection(pf)
>>>>>     tracerp25 = alld.cut_region(["grid['specific_scalar[0]'] > 0.25", "grid['z'] > 0.04"])
>>>>>     tp25extrema = tracerp25.quantities["Extrema"]("Density")
>>>>>     pc.add_phase_object(tracerp25,["zkpc","z-velocity","CellMassSolar"],weight=None,x_bins=50,y_bins=50,y_bounds = [-1e8,1.5e8],x_log=False,y_log=False)
>>>>>     pc.set_zlim(10000,1e10)
>>>>>     pc.save()
>>>>>     
>>>>>     i = i+1
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>> 
>>>> 
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>> 
>>> 
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>> 
>> 
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> 
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20131023/20787523/attachment.htm>