[yt-users] pickle

[Matthew Turk]

On Thu, Aug 11, 2011 at 11:41 AM, Elizabeth Tasker <taskere at mcmaster.ca> wrote:
> Hi Matt and Stephen,
>
> While typing out what Matt asked for, I inadvertently fixed the problem! It
> turns out that the equivalent to:
>
> -------> print(cloudcores[40]["Density"])
> yt : [INFO     ] 2011-08-11 13:39:59,985 Getting field Density from 2
> [  3.80336226e-23   4.31254788e-23   3.87927154e-23 ...,   1.26533774e-22
>  2.13753864e-22   2.74680682e-22]
>
>
> once pickled and de-pickled into 'core' is:
>
> -------> print(core[40][1]["Density"])
> yt : [INFO     ] 2011-08-11 13:38:48,531 Getting field Density from 2
> [  3.80336226e-23   4.31254788e-23   3.87927154e-23 ...,   1.26533774e-22
>  2.13753864e-22   2.74680682e-22]
>
>
> since
>
> In [29]: print core[40][0]
> -------> print(core[40][0])
> GT_BTAccel_256AMR4_PeHeat_sf5_SNe_0301
>
> -------> print(core[40][1])
> ExtractedRegion (GT_BTAccel_256AMR4_PeHeat_sf5_SNe_0301):
> _base_region=AMRRegion (GT_BTAccel_256AMR4_PeHeat_sf5_SNe_0301): center=[
> 16.  16.  16.], left_edge=[ 0.  0.  0.], right_edge=[ 32.  32.  32.]
>
> As a matter of interest, I couldn't get the yt-approved object storage to
> work. I just got:
>
> In [14]: pf.h.save_object("cloudcores", "myclouds")
> ---------------------------------------------------------------------------
> AttributeError                            Traceback (most recent call last)
>
> /1/home/taskere/yt/scripts/iyt in <module>()
> ----> 1
>     2
>     3
>     4
>     5
>
> AttributeError: 'NoneType' object has no attribute 'h'

This error looks like pf is None, which is a different issue...

>
> I tried also:
>
> In [15]: dd.save_object("cloudcores", "myclouds")
> ---------------------------------------------------------------------------
> NameError                                 Traceback (most recent call last)
>
> /1/home/taskere/yt/scripts/iyt in <module>()
> ----> 1
>     2
>     3
>     4
>     5
>
> /1/home/taskere/yt/yt/data_objects/data_containers.py in save_object(self,
> name, filename)
>   329         """
>   330         if filename is not None:
> --> 331             ds = shelve.open(filename, protocol=-1)
>   332             if name in ds:
>   333                 mylog.info("Overwriting %s in %s", name, filename)
>
> NameError: global name 'shelve' is not defined

Fixed in tip, thanks!

-Matt

>
> (I added in an 'import shelve' for good measure, but it didn't help).
>
> Finally, I tried:
>
> In [18]: cloudcores.save_object("myclouds", "cloudstored.cpkl")
> ---------------------------------------------------------------------------
> AttributeError                            Traceback (most recent call last)
>
> /1/home/taskere/yt/scripts/iyt in <module>()
> ----> 1
>     2
>     3
>     4
>     5
>
> AttributeError: 'dict' object has no attribute 'save_object'
>
>
> Thanks once again,
>
> Elizabeth
>
>
>
>
> Matthew Turk wrote:
>>
>> It might also be worthwhile to see what, exactly, core is ...
>>
>> print core
>>
>> Depending on how it is loaded/unloaded, it might be a tuple of the
>> parameter file, or might be something slightly other than you are
>> expecting.  This comes about because it tries really hard to make sure
>> that the parameter file doesn't disappear.
>>
>> -Matt
>>
>> On Thu, Aug 11, 2011 at 9:20 AM, Stephen Skory <s at skory.us> wrote:
>>
>>>
>>> Elizabeth,
>>>
>>> have you tried using the yt-approved object storage?
>>>
>>>
>>> http://yt.enzotools.org/doc/analyzing/objects.html?#storing-and-loading-objects
>>>
>>> On Thu, Aug 11, 2011 at 9:09 AM, Elizabeth Tasker <taskere at mcmaster.ca>
>>> wrote:
>>>
>>>>
>>>> Hi,
>>>>
>>>> (Another day a different problem!): I'd like to pickle the results from
>>>> extract_connected_sets, but I can't seem to retrieve what I pickle
>>>> properly.
>>>> I have:
>>>>
>>>> contours = dd.extract_connected_sets("NegEscapeVelocity", 1, 30.0, maxv,
>>>> log_space=False)
>>>> cloudcores = contours[1][0] #cores defined by contour 0
>>>>
>>>> file = open('testa.pickle','wb')
>>>> cPickle.dump(cloudcores,file,protocol=-1)
>>>> file.close()
>>>>
>>>> file = open('testa.pickle','rb')
>>>> core = cPickle.load(file)
>>>> file.close()
>>>>
>>>> But while:
>>>>
>>>> In [76]: cloudcores[40]["Density"]
>>>> Out[76]:
>>>> array([  3.80336226e-23,   4.31254788e-23,   3.87927154e-23, ...,
>>>>       1.26533774e-22,   2.13753864e-22,   2.74680682e-22])
>>>>
>>>>
>>>>
>>>> In [77]: core[40]["Density"]
>>>>
>>>> ---------------------------------------------------------------------------
>>>> KeyError                                  Traceback (most recent call
>>>> last)
>>>>
>>>> /1/home/taskere/yt/scripts/iyt in <module>()
>>>> ----> 1
>>>>    2
>>>>    3
>>>>    4
>>>>    5
>>>>
>>>> KeyError: 40
>>>>
>>>>
>>>> Is it not possible to pickle a contour object or is there something else
>>>> I
>>>> have to do when un-pickling?
>>>>
>>>> Thank you,
>>>>
>>>> Elizabeth
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>
>>> --
>>> Stephen Skory
>>> s at skory.us
>>> http://stephenskory.com/
>>> 510.621.3687 (google voice)
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>