[yt-dev] Native field discussion

Tue May 7 10:05:26 PDT 2013

On Tue, May 7, 2013 at 12:13 PM, Casey W. Stark <caseywstark at gmail.com> wrote:
> Hey Matt.
>
> Excellent! It's a considerable change, so we should dread it a bit, but
> really glad you like the idea too.
>
> About where the conversion should happen, I like the idea to return any
> universal fields in cgs by default. Operations between "code" fields
> ("mycode_density") would return in code units and ops between universal
> fields ("density") would return in cgs. That means most users won't see a
> change too. Mixing units is mostly straightforward. I think the only gotcha
> is for add and sub, where we return in the left object's units.

That sounds good to me.

>
>> How do we handle the fact
> that converting "code_density" to cgs is different for different
> arrays, depending on the code?  i.e., if I have two "code_density"
> arrays from a code that stores them in comoving code units, how do we
> distinguish between those two things during division, say?
>
> I'm not sure I'm following here, but is this a case where density field a
> has different units from density field b? If so, I think we have to define
> separate units for the code densities. Here's a quick Nyx example...

Not exactly, but close -- think of having two different parameter
files from the same simulation open at the same time.  If the sim
stores internally in comoving units, the native->propercgs will vary
between those two, but the registered units might be the same name,
for instance "enzo_density" or something.

>
> unit_registry.add("nyx_cm_density", "Msun/Mpccm**3")
> unit_registry.add("nyx_prop_density", "g/cm**3")
>
> add_field("nyx_density", units="nyx_cm_density")
> add_field("nyx_hi_density", units="nyx_prop_density")
>
> dd["nyx_density"] / dd["nyx_hi_density"]    (should return dimensionless
> YTArray)
>
> - Casey
>
>
>
> On Tue, May 7, 2013 at 8:18 AM, Matthew Turk <matthewturk at gmail.com> wrote:
>>
>> Hi Casey and Nathan,
>>
>> Last night when I read this I felt a bit of dread in my stomach,
>> worrying that this would make things a lot more complex and difficult.
>>  But I have to say, now having thought it through and read your
>> pastes, reflected on the contents of the YTEP ... I think this is
>> beautiful and elegant.  Very nicely thought out.
>>
>> I completely agree that -- particularly in 3.0 -- we no longer need to
>> perform any conversions at IO time, and instead we should be pushing
>> this to the places where the data will be presented to individual
>> users.  This will require some thought about where we want this
>> conversion to take place (i.e., at the display layer?) but it should
>> be completely workable to do so.  The ensure_cgs decorator is a great
>> way to manage this.
>>
>> I'm +1 on this, and can help with the technical details.  As it
>> stands, all unit conversions happen in a single place in the code, so
>> we can simply remove the items there and instead just wrap them in the
>> appropriate "code" units.  We'll need to figure out the specific
>> semantics of that but it should be okay.  How do we handle the fact
>> that converting "code_density" to cgs is different for different
>> arrays, depending on the code?  i.e., if I have two "code_density"
>> arrays from a code that stores them in comoving code units, how do we
>> distinguish between those two things during division, say?
>>
>> -Matt
>>
>> On Mon, May 6, 2013 at 6:50 PM, Casey W. Stark <caseywstark at gmail.com>
>> wrote:
>> > Hey Nathan.
>> >
>> > Thanks for the reminder about `ensure_cgs`. Definitely helps to see the
>> > pseudocode too. Users could also write out the array.convert_to(...) in
>> > the
>> > field function for whatever obscure units. I guess this is similar to
>> > the
>> > fields with the units tacked on to the name, but much cleaner.
>> >
>> > I think making the code to universal field mapping more explicit will be
>> > very helpful. I've been confused about where some derived fields come
>> > from
>> > many times!
>> >
>> >
>> > On Mon, May 6, 2013 at 3:39 PM, Nathan Goldbaum <goldbaum at ucolick.org>
>> > wrote:
>> >>
>> >> Hey Casey,
>> >>
>> >> That's pretty much what I was thinking, although I wanted to add a
>> >> little
>> >> bit of syntactic sugar so the CGS conversion happens automatically:
>> >>
>> >> http://paste.yt-project.org/show/3443/
>> >>
>> >> The ensure_cgs decorator would simply call convert_to_cgs on the
>> >> returned
>> >> YTArray.  The nice thing about this is the flow of data is clear,
>> >> things
>> >> come in to the field definition code units and leave in CGS.  This also
>> >> eliminates the need for the unit conversion functions that are
>> >> scattered
>> >> throughout the frontends and universal_fields.py.
>> >>
>> >> This has the nice feature that each of the code frontends will need to
>> >> explicitly define mappings to the CGS fields (like Density) needed by
>> >> universal fields.
>> >>
>> >> -Nathan
>> >>
>> >>
>> >> On Mon, May 6, 2013 at 3:33 PM, Casey W. Stark <caseywstark at gmail.com>
>> >> wrote:
>> >>>
>> >>> Hey yt.
>> >>>
>> >>> Nathan and I had a cool idea as an extension of the units work, but I
>> >>> am
>> >>> hitting a yt internals knowledge wall writing a YTEP. I would like
>> >>> some
>> >>> input on technical details or general feelings before I move ahead
>> >>> with the
>> >>> write-up.
>> >>>
>> >>> One of the things mentioned in the units YTEP is handling code units.
>> >>> Basically each frontend static output should register the code units.
>> >>> In Nyx
>> >>> for instance, we would do something like registering "code_length" as
>> >>> Mpccm,
>> >>> "code_mass" as Msun, etc. Then any field from the dataset can be
>> >>> loaded in
>> >>> code units with something like dd["density"].in_units("code_density").
>> >>>
>> >>> This got us thinking -- why load native fields in CGS now? Instead, we
>> >>> can define native fields exactly as they are on disk and users who
>> >>> want code
>> >>> units will never have to convert by hand again. If frontend developers
>> >>> define the fields and units on disk, and some mapping to the universal
>> >>> fields, we shouldn't have to define or convert units anywhere else.
>> >>>
>> >>> I made a quick and dirty example of what we're thinking here:
>> >>> http://paste.yt-project.org/show/3441/
>> >>>
>> >>> So, is something like this a good idea? Is it realistic? Would this
>> >>> make
>> >>> for a simpler field info container? Nathan, is there anything I
>> >>> missed?
>> >>>
>> >>> - Casey
>> >>>
>> >>> _______________________________________________
>> >>> yt-dev mailing list
>> >>> yt-dev at lists.spacepope.org
>> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>>
>> >>
>> >>
>> >> _______________________________________________
>> >> yt-dev mailing list
>> >> yt-dev at lists.spacepope.org
>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>
>> >
>> >
>> > _______________________________________________
>> > yt-dev mailing list
>> > yt-dev at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>