<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Feb 18, 2017 at 9:28 PM, Jason Galyardt <span dir="ltr"><<a href="mailto:jason.galyardt@gmail.com" target="_blank">jason.galyardt@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div><div><div>Hi Nathan,<br><br></div><div>For a single profile plot, it's straightforward to save it as a yt dataset and reload it later:<br></div><div><br></div><div>#### Begin ####<br></div><div>ds = yt.load('<span class="gmail-m_-8307507391008841415gmail-s">GasSloshing/sloshing_<wbr>nomag2_hdf5_plt_cnt_0150')<br></span><span class="gmail-m_-8307507391008841415gmail-s">ad = ds.all_data()<br>prof = yt.create_profile(ad,["z"],<wbr>fields=["velocity_z"], n_bins=100, logs={"z":False})<br></span></div><div><span class="gmail-m_-8307507391008841415gmail-s">fn = prof.save_as_dataset()<br></span></div><div><span class="gmail-m_-8307507391008841415gmail-s"># restore...<br></span></div><div><span class="gmail-m_-8307507391008841415gmail-s">prof_ds = yt.load(fn)<br></span></div><div><span class="gmail-m_-8307507391008841415gmail-s">prof_ds_plot = yt.ProfilePlot.from_profiles([<wbr>prof_ds.profile], y_log=<br> {"velocity_z":False})<br></span></div><div>#### End ####<br><br></div><div>The above workflow breaks down when one has several different profiles to generate from a time series. One can manage it via the filenames, but this seems prone to error and inefficient.<br><br></div>Here's a sketch of what I'd like to do, with the time series processing removed for clarity:<br><br></div><div>#### Begin ####<br></div><div>import yt<br></div><div>import gzip<br></div><div>import cPickle<br></div>ds = yt.load('<span class="gmail-m_-8307507391008841415gmail-s">GasSloshing/sloshing_<wbr>nomag2_hdf5_plt_cnt_0150')<br></span></div><span class="gmail-m_-8307507391008841415gmail-s">ad = ds.all_data()<br>prof = yt.create_profile(ad,["z"],<wbr>fields=["velocity_z"], n_bins=100, logs={"z":False})<br></span></div><div><span class="gmail-m_-8307507391008841415gmail-s">fname = </span><span class="gmail-m_-8307507391008841415gmail-s">'test_pkl.gz'<br><br></span></div><div><span class="gmail-m_-8307507391008841415gmail-s">## Here's where things break down ##<br></span></div><div><span class="gmail-m_-8307507391008841415gmail-s"># The following does NOT work because prof is a function object<br></span></div><div><span class="gmail-m_-8307507391008841415gmail-s">with gzip.open(fname,'wb') as f:<br></span></div><span class="gmail-m_-8307507391008841415gmail-s"> cPickle.dump(prof, f)<br></span></div><span class="gmail-m_-8307507391008841415gmail-s"> f.close()<br></span><div><div><span class="gmail-m_-8307507391008841415gmail-s"># TypeError: can't pickle function objects<br></span></div></div></div></blockquote><div><br></div><div>Yes, right now profile objects can't be pickled, as you've discovered. I don't think it was ever explicitly designed to be pickleable, so that's why it doesn't work for you here.</div><div><br></div><div>I don't think there's anything preventing this from happening, it's just that the code and tests need to be written to support it.</div><div><br></div><div>Adding support for this means implementing the pickle object serialization protocol (e.g. __reduce__ and __setstate__) on the ProfileND class.</div><div><br></div><div>If you don't want to take a stab at doing this yourself, please open an issue about it on our bug tracker so we don't lose track:</div><div><br></div><div><a href="https://bitbucket.org/yt_analysis/yt/issues/new">https://bitbucket.org/yt_analysis/yt/issues/new</a><br></div><div><br></div><div>For now the suggested way to serialize yt data objects is to use the ytdata frontend.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><span class="gmail-m_-8307507391008841415gmail-s"></span><div><div><div><div><div><div class="gmail_extra"><br></div><div class="gmail_extra"># Resuming...<br></div># From here, I'd like to load the profile object from the pickle file. If the profile <br></div><div># prof were a regular object, I would be able to do the following:<br></div><div>with gzip.open(fname, 'rb') as f:<br></div><div> prof_pkl = cPickle.load(f)<br></div><div> f.close()<br></div><div># With the time series processing, there would be several profiles available <br></div><div># for plotting on the same set of axes, but for the purposes of this example, <br></div><div># I'm only including the one profile loaded above:<br></div><div>prof_plot = yt.ProfilePlot.from_profiles([<wbr>prof_pkl], y_log={"velocity_z":False}, <br> labels=['aa'])<br></div><div>#### End ####<br><br></div><div>As a stop gap, I'll just save all the critical bits of the profile function object (e.g. prof.field_data, prof.x, prof.x_bins, etc.) to a dictionary, then pickle the dictionary.<br></div><div><div class="gmail_extra">Is there a better way?<br><br></div><div class="gmail_extra">Thanks,<br></div><div class="gmail_extra">Jason<br><br></div><div class="gmail_extra"><br></div><div class="gmail_extra"><div class="gmail_quote">On Sat, Feb 18, 2017 at 2:05 PM, <span dir="ltr"><<a href="mailto:yt-users-request@lists.spacepope.org" target="_blank">yt-users-request@lists.<wbr>spacepope.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Send yt-users mailing list submissions to<br>
<a href="mailto:yt-users@lists.spacepope.org" target="_blank">yt-users@lists.spacepope.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" rel="noreferrer" target="_blank">http://lists.spacepope.org/lis<wbr>tinfo.cgi/yt-users-spacepope.<wbr>org</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:yt-users-request@lists.spacepope.org" target="_blank">yt-users-request@lists.spacepo<wbr>pe.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:yt-users-owner@lists.spacepope.org" target="_blank">yt-users-owner@lists.spacepope<wbr>.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of yt-users digest..."<br>
<br>Today's Topics:<br>
<br>
1. Profile data (Jason Galyardt)<br>
2. Re: Profile data (Nathan Goldbaum)<span class="gmail-"><br>
<br><br>---------- Forwarded message ----------<br>From: Jason Galyardt <<a href="mailto:jason.galyardt@gmail.com" target="_blank">jason.galyardt@gmail.com</a>><br>To: Discussion of the yt analysis package <<a href="mailto:yt-users@lists.spacepope.org" target="_blank">yt-users@lists.spacepope.org</a>><br>Cc: <br>Date: Fri, 17 Feb 2017 15:58:27 -0500<br>Subject: [yt-users] Profile data<br></span><div dir="ltr"><span class="gmail-"><div><div>Dear yt Users, <br><br></div>Does anyone know of a way to export the data of a profile object to a byte stream? Specifically, I'd like to be able to store several byte streams from several different simulation files into a single dictionary object (for example), and then pickle the dictionary. I'd like to then unpickle the dictionary object at a later time for plotting, etc. <br><br></div>I have figured out a way to do this, but it involves saving each profile to a temporary file, reading the temp file into a byte stream, and then pickling the byte stream; then to reload, I would have to unpickle the byte stream, write it to a temporary file, then use yt.load() to get the profile object back. This works, but boy, it's inefficient. Dealing with a whole bunch of separate files generated by profile.save_as_dataset() is inefficient in different way. Any ideas on streamlining this workflow?<br></span><div><div><br></div><span class="gmail-"><div>Thanks,<br></div><div>Jason<br><br>------<br></div><div>Jason Galyardt<br></div><div>University of Georgia<br><br></div></span></div></div><div><div class="gmail-h5">
<br><br>---------- Forwarded message ----------<br>From: Nathan Goldbaum <<a href="mailto:nathan12343@gmail.com" target="_blank">nathan12343@gmail.com</a>><br>To: Discussion of the yt analysis package <<a href="mailto:yt-users@lists.spacepope.org" target="_blank">yt-users@lists.spacepope.org</a>><br>Cc: <br>Date: Fri, 17 Feb 2017 15:01:21 -0600<br>Subject: Re: [yt-users] Profile data<br><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Feb 17, 2017 at 2:58 PM, Jason Galyardt <span dir="ltr"><<a href="mailto:jason.galyardt@gmail.com" target="_blank">jason.galyardt@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div>Dear yt Users, <br><br></div>Does anyone know of a way to export the data of a profile object to a byte stream? Specifically, I'd like to be able to store several byte streams from several different simulation files into a single dictionary object (for example), and then pickle the dictionary. I'd like to then unpickle the dictionary object at a later time for plotting, etc. <br><br></div>I have figured out a way to do this, but it involves saving each profile to a temporary file, reading the temp file into a byte stream, and then pickling the byte stream; then to reload, I would have to unpickle the byte stream, write it to a temporary file, then use yt.load() to get the profile object back. This works, but boy, it's inefficient. Dealing with a whole bunch of separate files generated by profile.save_as_dataset() is inefficient in different way. Any ideas on streamlining this workflow?</div></blockquote><div><br></div><div>I'm not sure I fully understand what you're trying to do. A code example or an outline if a code example with the part you're confused about left to be filled in would help.</div><div><br></div><div>Why not just save the raw profile data for the fields you're interested in, e.g. profile[field], which will be a numpy array. Do you need other data that's defined on the profile object?</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><br></div><div>Thanks,<br></div><div>Jason<br><br>------<br></div><div>Jason Galyardt<br></div><div>University of Georgia<br><br></div></div></div>
<br>______________________________<wbr>_________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org" target="_blank">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" rel="noreferrer" target="_blank">http://lists.spacepope.org/lis<wbr>tinfo.cgi/yt-users-spacepope.o<wbr>rg</a><br>
<br></blockquote></div><br></div></div>
<br>______________________________<wbr>_________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org" target="_blank">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" rel="noreferrer" target="_blank">http://lists.spacepope.org/lis<wbr>tinfo.cgi/yt-users-spacepope.<wbr>org</a><br>
<br></div></div></blockquote></div><br></div></div></div></div></div></div></div></div></div>
<br>______________________________<wbr>_________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" rel="noreferrer" target="_blank">http://lists.spacepope.org/<wbr>listinfo.cgi/yt-users-<wbr>spacepope.org</a><br>
<br></blockquote></div><br></div></div>