[yt-users] Fwd: [Yt-dev] saving derived fields

[Matthew Turk]

Andrew submitted a first pass at this:

https://bitbucket.org/yt_analysis/yt/pull-request/416/saving-derived-fields-to-disk/

I think I'm on board with John's comment: this should be broadly
available.  Great work, Andrew!

On Tue, Feb 5, 2013 at 4:39 PM, Matthew Turk <matthewturk at gmail.com> wrote:
> Hey Andrew et al,
>
> In the intervening time, I've had a chance ot think about this more
> too.  I think that Sam's idea of using the write_gdf is a good one.
> In fact, I would go so far as to say that this is the *right* way to
> do it, and to allow individuals to affiliate a "backup" GDF file with
> the dataset. Then inside the IO functionality, if a given field does
> *not* exist, it could simply be read from the GDF file.
>
> This can likely be done already by generating the derived fields and
> writing out a GDF file, as noted in an earlier thread.  But longer
> term, using the Stream frontend in a more complex way, we can apply
> fallbacks -- have a "Merged" output that includes the Orion data *and*
> the GDF, but the two files exist separately.  This would require not
> using load_unifrom_grid or load_amr_grids, but instead creating a
> Stream hierarchy manually.
>
> -Matt
>
> On Tue, Feb 5, 2013 at 12:53 PM, Andrew Myers <atmyers at berkeley.edu> wrote:
>> Hi Sam,
>>
>> Thanks for the info! I'll give it a try and report back if I get something
>> working.
>>
>> -Andrew
>>
>>
>> On Tue, Feb 5, 2013 at 12:12 PM, Sam Skillman <samskillman at gmail.com> wrote:
>>>
>>> Oh man, that Sam Skillman from a couple of years ago really dropped the
>>> ball.  I did not end up giving it a go or report back!
>>>
>>> That said, I think Matt's suggestion still holds up.
>>>
>>> One other option would be to use (or modify) the write_to_gdf
>>> functionality, though this would duplicate the data.
>>>
>>> So to reuse some bits...
>>> If you give this a shot and test it, let us know how it goes, and feel
>>> free to pull request if it works out.
>>>
>>> Sam
>>>
>>>
>>> On Tue, Feb 5, 2013 at 12:50 PM, Andrew Myers <atmyers at berkeley.edu>
>>> wrote:
>>>>
>>>> Hello YT users,
>>>>
>>>> A couple of years ago, Sam Skillman (email chain attached) asked about
>>>> support for saving derived fields to the disk, so we don't have to
>>>> re-compute expensive fields over and over again. Did this functionality ever
>>>> make it into yt? I'm working on something where this would be very useful,
>>>> so if not I might have a go at implementing myself.
>>>>
>>>> Thanks,
>>>> Andrew Myers
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: Sam Skillman <samskillman at gmail.com>
>>>> Date: Tue, Jan 11, 2011 at 9:40 AM
>>>> Subject: [Yt-dev] saving derived fields
>>>> To: yt-dev at lists.spacepope.org
>>>>
>>>>
>>>> Hi all,
>>>>
>>>> I've started dealing with a lot of time-consuming derived fields and am
>>>> wondering if there is already support for writing derived fields back to the
>>>> raw data files (e.g. .cpu files).  If not, any thoughts on doing so?
>>>>
>>>> Sam
>>>>
>>>> _______________________________________________
>>>> Yt-dev mailing list
>>>> Yt-dev at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>