[yt-users] MemoryError when using BinnedProfile1D

[Yuan Li]

Hi Matt,

Thank you! Setting lazy_reader helped a lot. I don't know why I did not try
that before!

Thanks,
Yuan


On Fri, Jul 19, 2013 at 3:44 PM, Matthew Turk <matthewturk at gmail.com> wrote:

> Hi Yuan,
>
> Oh, somehow I completely spaced on this, but BP1D is by default not memory
> conservative but a_p_s is.
>
> Supply the argument:
>
> lazy_reader=True
>
> when you create the BP1D, and it should fix it.
>
> Sorry about that!
>
> Matt
> On Jul 19, 2013 3:43 PM, "Yuan Li" <bear0980 at gmail.com> wrote:
>
>> Hi Matt,
>>
>> I thought about the first one too. It did not seem to matter whether or
>> not I cut the region.
>>
>> yt instinfo gives me this:
>> The current version of the code is:
>> e3e4f4578504
>>
>> I have just updated yt again, and I am still getting the same error
>> message. I did manage to make the plots with 64 gb memory, so I guess it
>> really was just a memory issue. It would be great if BinnedProfile1D could
>> be more memory conservative, but for now, I will use add_profile_sphere
>> whenever possible.
>>
>>
>> Thanks!
>> Yuan
>>
>>
>>
>> On Fri, Jul 19, 2013 at 8:16 AM, Matthew Turk <matthewturk at gmail.com>wrote:
>>
>>> Hi Yuan,
>>>
>>> On Thu, Jul 18, 2013 at 9:43 PM, Yuan Li <bear0980 at gmail.com> wrote:
>>> > Hi yt users,
>>> >
>>> > I have been having trouble using BinnedProfile1D. I am using the same
>>> script
>>> > that seemed to be working before. Now it is giving me "MemoryError":
>>> > http://paste.yt-project.org/show/A67caGKsv6mFqA83MZoZ/
>>> >
>>> > I can still use it on a low resolution run without a problem.
>>> >
>>> >
>>> > I am not sure why it needs so much memory (I am already asking for 30
>>> gb of
>>> > memory). If I do a similar analysis with plotcollection and
>>> > add_profile_sphere for the same data, I do not need to ask for much
>>> memory
>>> > at all. I thought BinnedProfile1D would need less memory than
>>> > add_profile_sphere. Please let me know if I was wrong and the job
>>> simply
>>> > requires a lot of memory.
>>> >
>>> > What I am trying to do is just very basic analysis:
>>> > ...
>>> > sp=pf.h.sphere([0.5, 0.5, 0.5], rmax/pf['kpc'])
>>> > sp_in=sp.cut_region(["grid['Theta']>0.6"])
>>> > prof=BinnedProfile1D(sp_in, 128, "Radiuskpc", 1e-2,rmax)
>>> > r=prof["Radiuskpc"]
>>> > prof.add_fields("Temperature",weight="CellMassMsun")
>>>
>>> Hm, this looks okay to me.  The only two things that I think could be a
>>> problem:
>>>
>>>  * Have you tried it on the sp object instead of sp_in?  It may be
>>> that the cut_region is not operating in a memory conservative way.
>>>  * Which version are you on?  (yt instinfo should tell you)  A month
>>> or so ago we identified (thanks to Mike Kuhlen and Nathan Goldbaum) a
>>> way to dramatically reduce memory used in radius fields.  That may be
>>> relevant here.
>>>
>>> -Matt
>>>
>>> > ...
>>> >
>>> >
>>> > It usually fails at the temperature part. Please let me know if you
>>> have any
>>> > ideas or suggestions.
>>> >
>>> > Thank you!
>>> > Yuan
>>> >
>>> > _______________________________________________
>>> > yt-users mailing list
>>> > yt-users at lists.spacepope.org
>>> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>> >
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20130722/e29a55f1/attachment.html>