[Yt-dev] field_info refactor design

[Matthew Turk]

Hi Casey,

I'm back from vacation and ready to start thinking about this again.  :)

On Thu, Jul 14, 2011 at 12:31 PM, Casey W. Stark <caseywstark at gmail.com> wrote:
> Hi Matt.
> Thanks for the reply. And don't worry about me getting stuck - I've been
> developing frontend stuff on another branch.
> That's a good point - I hadn't thought of data and derived fields with the
> same name. I think I'm confused about how this is supposed to fit together
> now. Here's an example from the current version of the code.
> In the field_info_container module, you have FieldInfoContainer the class,
> and FieldInfo, an instantiation of FieldInfoContainer.

Yup, and FieldInfo is the container for *all* the derived fields that
are meant to be universal.  This is stuff like cosmological over
density, for instance, which broadly speaking has a unique method of
generation independent of code.  (And where the gas relationships are
fully determinable from the parameter file, this class of field can
include things like pressure, sound speed, etc.)  In theory the idea
of a set of universal fields works out better than in practice, for
the most part, simply because of such divergent problem domains.

> Then in the enzo
> frontend fields module, we have:
> EnzoFieldInfo = FieldInfoContainer.create_with_fallback(FieldInfo)
> add_field = EnzoFieldInfo.add_field
> KnownEnzoFields = FieldInfoContainer()
> add_enzo_field = KnownEnzoFields.add_field
> I think I wrote this when I didn't exactly understand the design. I believe
> it should be...
> derivable_enzo_fields = FieldInfo
> add_field = derivable_enzo_fields.add_field
> known_enzo_fields =
> FieldInfoContainer.create_with_fallback(derivable_enzo_fields)
> add_enzo_field = known_enzo_fields.add_field

This sounds good -- an important semantic difference.  Note that one
reason we keep add_field around is for legacy code which uses it,
particularly in users' ~/.yt/my_plugins.py file.

> The part that is confusing me is in the frontend StaticOutput classes. Here
> it is in enzo data_structures.
> class EnzoStaticOutput(StaticOutput):
>     ...
>     _fieldinfo_fallback = EnzoFieldInfo
>     _fieldinfo_known = KnownEnzoFields
>     ...
>     def __init__(...):
>         ...
>         self.field_info =
> FieldInfoContainer.create_with_fallback(self._fieldinfo_fallback)
> So it looks like _fieldinfo_known is never used and we make a new
> FieldInfoContainer instead. In this case, how does field_info know which
> fields can be in an enzo data file and which ones exist in the current
> staticoutput?

Ah, but fieldinfo_known *is* used, but in the base class.  Check
yt/data_objects/hierarchy.py .  What happens is that known_fields is
queried and if the field is found, it is added to the pf.field_info.
This way the definition is grabbed from the list of known items.

> The way I thought about it, each StaticOutput object would have a copy of
> that frontend's known_field object and customize the fields during __init__.
> Something like...
> self.field_info = self._fieldinfo_known.setup_for_output(self)
> I'll try to stare at the code some more, but if anyone has experience with
> this part of yt, please let me know what you think and I can code it up.
> Best,
> Casey

I think that's sort of how I aimed to implement it, too.

Talk to you soon,

Matt

>
> On Mon, Jul 11, 2011 at 8:55 AM, Matthew Turk <matthewturk at gmail.com> wrote:
>>
>> Hi Casey,
>>
>> Sorry for the brief reply; I'm on vacation but have a handful of
>> minutes to browse my email and I don't want you to get stuck too long.
>>  I don't think I understand the issue -- I think the pf.field_info
>> object is supposed to contain all of the data fields, with a fallback
>> for derived.  The reason for this is so that adding a derived field
>> after the instantiation of a parameter file will allow it to be
>> accessed.  i.e.,
>>
>> pf = load(...)
>> add_field("Something")
>>
>> "Something" won't be available if the fields are fixed at
>> instantiation time with no fallback.  One of the reasons we have the
>> "known in data" and "derived" separation is so that if for instance
>> the code can optionally output a field (like "Dark_Matter_Density") we
>> should be able to use it, but if it doesn't, then it should make a new
>> derived field.  Does that make sense?
>>
>> However, it sounds like something is more broadly wrong than just the
>> fallbacks -- likely my fault somewhere along the line.  If you can
>> post a slightly more detailed design description maybe I can
>> understand better and we can then adopt your method.
>>
>> -Matt
>>
>> On Wed, Jul 6, 2011 at 5:46 PM, Casey W. Stark <caseywstark at gmail.com>
>> wrote:
>> > Hello yt.
>> > I'm working on the deliberate_fields branch and came up with a design
>> > idea.
>> > I want to check in with yt-dev before implementing it.
>> > I was trying to get the new design to pass tests, but could not get it
>> > working. It looks like pf.field_info is fine during the first test and
>> > is
>> > empty when the second test starts. I suspect this issue is related to
>> > the
>> > new fallback design, but I could not track it down.
>> > The new field_info design gives FieldInfoContainer a fallback attribute
>> > that
>> > should be another FieldInfoContainer. The idea is that a parent
>> > FieldInfoContainer contains all the fields from data, and the fallback
>> > attribute contains all of the derivable fields. Fields in both of these
>> > objects are instances of
>> > `yt.data_objects.field_info_container.DerivedField`, created with the
>> > add_field method in a frontend fields module.
>> > I thought it would be easier to have a FieldInfoContainer that contains
>> > all
>> > fields (for a given frontend) and is aware of which fields are from data
>> > and
>> > which are derivable. I think you can get the same behavior as the
>> > fallback
>> > design without modifying the dict behavior. When defining the frontend
>> > fields, you add fields that might be in data files with
>> > code_field_info.add_data_field and you add fields that could be derived
>> > given all dependences with code_field_info.add_derivable_field. When a
>> > pf is
>> > loaded, prune the default field_info to contain only the available
>> > fields.
>> > Of course, I'm pretty new to the yt code, so I don't want to break any
>> > behavior before 3.0. I would really appreciate any feedback, especially
>> > if
>> > you know this would create problems. I will stash my changes for now and
>> > come back to it.
>> > Best,
>> > Casey
>> > _______________________________________________
>> > Yt-dev mailing list
>> > Yt-dev at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> >
>> _______________________________________________
>> Yt-dev mailing list
>> Yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
> _______________________________________________
> Yt-dev mailing list
> Yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>