[yt-users] Profile data

Nathan Goldbaum nathan12343 at gmail.com
Mon Feb 20 08:41:48 PST 2017


On Sat, Feb 18, 2017 at 9:28 PM, Jason Galyardt <jason.galyardt at gmail.com>
wrote:

> Hi Nathan,
>
> For a single profile plot, it's straightforward to save it as a yt dataset
> and reload it later:
>
> #### Begin ####
> ds = yt.load('GasSloshing/sloshing_nomag2_hdf5_plt_cnt_0150')
> ad = ds.all_data()
> prof = yt.create_profile(ad,["z"],fields=["velocity_z"], n_bins=100,
> logs={"z":False})
> fn = prof.save_as_dataset()
> # restore...
> prof_ds = yt.load(fn)
> prof_ds_plot = yt.ProfilePlot.from_profiles([prof_ds.profile], y_log=
>       {"velocity_z":False})
> #### End ####
>
> The above workflow breaks down when one has several different profiles to
> generate from a time series. One can manage it via the filenames, but this
> seems prone to error and inefficient.
>
> Here's a sketch of what I'd like to do, with the time series processing
> removed for clarity:
>
> #### Begin ####
> import yt
> import gzip
> import cPickle
> ds = yt.load('GasSloshing/sloshing_nomag2_hdf5_plt_cnt_0150')
> ad = ds.all_data()
> prof = yt.create_profile(ad,["z"],fields=["velocity_z"], n_bins=100,
> logs={"z":False})
> fname = 'test_pkl.gz'
>
> ## Here's where things break down ##
> # The following does NOT work because prof is a function object
> with gzip.open(fname,'wb') as f:
>     cPickle.dump(prof, f)
>     f.close()
> # TypeError: can't pickle function objects
>

Yes, right now profile objects can't be pickled, as you've discovered. I
don't think it was ever explicitly designed to be pickleable, so that's why
it doesn't work for you here.

I don't think there's anything preventing this from happening, it's just
that the code and tests need to be written to support it.

Adding support for this means implementing the pickle object serialization
protocol (e.g. __reduce__ and __setstate__) on the ProfileND class.

If you don't want to take a stab at doing this yourself, please open an
issue about it on our bug tracker so we don't lose track:

https://bitbucket.org/yt_analysis/yt/issues/new

For now the suggested way to serialize yt data objects is to use the ytdata
frontend.


>
> # Resuming...
> # From here, I'd like to load the profile object from the pickle file. If
> the profile
> # prof were a regular object, I would be able to do the following:
> with gzip.open(fname, 'rb') as f:
>     prof_pkl = cPickle.load(f)
>     f.close()
> # With the time series processing, there would be several profiles
> available
> # for plotting on the same set of axes, but for the purposes of this
> example,
> # I'm only including the one profile loaded above:
> prof_plot = yt.ProfilePlot.from_profiles([prof_pkl],
> y_log={"velocity_z":False},
>       labels=['aa'])
> #### End ####
>
> As a stop gap, I'll just save all the critical bits of the profile
> function object (e.g. prof.field_data, prof.x, prof.x_bins, etc.) to a
> dictionary, then pickle the dictionary.
> Is there a better way?
>
> Thanks,
> Jason
>
>
> On Sat, Feb 18, 2017 at 2:05 PM, <yt-users-request at lists.spacepope.org>
> wrote:
>
>> Send yt-users mailing list submissions to
>>         yt-users at lists.spacepope.org
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>>         http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>> or, via email, send a message with subject or body 'help' to
>>         yt-users-request at lists.spacepope.org
>>
>> You can reach the person managing the list at
>>         yt-users-owner at lists.spacepope.org
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of yt-users digest..."
>>
>> Today's Topics:
>>
>>    1. Profile data (Jason Galyardt)
>>    2. Re: Profile data (Nathan Goldbaum)
>>
>>
>> ---------- Forwarded message ----------
>> From: Jason Galyardt <jason.galyardt at gmail.com>
>> To: Discussion of the yt analysis package <yt-users at lists.spacepope.org>
>> Cc:
>> Date: Fri, 17 Feb 2017 15:58:27 -0500
>> Subject: [yt-users] Profile data
>> Dear yt Users,
>>
>> Does anyone know of a way to export the data of a profile object to a
>> byte stream? Specifically, I'd like to be able to store several byte
>> streams from several different simulation files into a single dictionary
>> object (for example), and then pickle the dictionary. I'd like to then
>> unpickle the dictionary object at a later time for plotting, etc.
>>
>> I have figured out a way to do this, but it involves saving each profile
>> to a temporary file, reading the temp file into a byte stream, and then
>> pickling the byte stream; then to reload, I would have to unpickle the byte
>> stream, write it to a temporary file, then use yt.load() to get the profile
>> object back. This works, but boy, it's inefficient. Dealing with a whole
>> bunch of separate files generated by profile.save_as_dataset() is
>> inefficient in different way. Any ideas on streamlining this workflow?
>>
>> Thanks,
>> Jason
>>
>> ------
>> Jason Galyardt
>> University of Georgia
>>
>>
>>
>> ---------- Forwarded message ----------
>> From: Nathan Goldbaum <nathan12343 at gmail.com>
>> To: Discussion of the yt analysis package <yt-users at lists.spacepope.org>
>> Cc:
>> Date: Fri, 17 Feb 2017 15:01:21 -0600
>> Subject: Re: [yt-users] Profile data
>>
>>
>> On Fri, Feb 17, 2017 at 2:58 PM, Jason Galyardt <jason.galyardt at gmail.com
>> > wrote:
>>
>>> Dear yt Users,
>>>
>>> Does anyone know of a way to export the data of a profile object to a
>>> byte stream? Specifically, I'd like to be able to store several byte
>>> streams from several different simulation files into a single dictionary
>>> object (for example), and then pickle the dictionary. I'd like to then
>>> unpickle the dictionary object at a later time for plotting, etc.
>>>
>>> I have figured out a way to do this, but it involves saving each profile
>>> to a temporary file, reading the temp file into a byte stream, and then
>>> pickling the byte stream; then to reload, I would have to unpickle the byte
>>> stream, write it to a temporary file, then use yt.load() to get the profile
>>> object back. This works, but boy, it's inefficient. Dealing with a whole
>>> bunch of separate files generated by profile.save_as_dataset() is
>>> inefficient in different way. Any ideas on streamlining this workflow?
>>>
>>
>> I'm not sure I fully understand what you're trying to do. A code example
>> or an outline if a code example with the part you're confused about left to
>> be filled in would help.
>>
>> Why not just save the raw profile data for the fields you're interested
>> in, e.g. profile[field], which will be a numpy array. Do you need other
>> data that's defined on the profile object?
>>
>>
>>> Thanks,
>>> Jason
>>>
>>> ------
>>> Jason Galyardt
>>> University of Georgia
>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20170220/db6c0581/attachment.html>


More information about the yt-users mailing list