[yt-dev] Native field discussion

Tue May 7 09:13:34 PDT 2013

Hey Matt.

Excellent! It's a considerable change, so we should dread it a bit, but
really glad you like the idea too.

About where the conversion should happen, I like the idea to return any
universal fields in cgs by default. Operations between "code" fields
("mycode_density") would return in code units and ops between universal
fields ("density") would return in cgs. That means most users won't see a
change too. Mixing units is mostly straightforward. I think the only gotcha
is for add and sub, where we return in the left object's units.

> How do we handle the fact
that converting "code_density" to cgs is different for different
arrays, depending on the code?  i.e., if I have two "code_density"
arrays from a code that stores them in comoving code units, how do we
distinguish between those two things during division, say?

I'm not sure I'm following here, but is this a case where density field a
has different units from density field b? If so, I think we have to define
separate units for the code densities. Here's a quick Nyx example...

unit_registry.add("nyx_cm_density", "Msun/Mpccm**3")
unit_registry.add("nyx_prop_density", "g/cm**3")

add_field("nyx_density", units="nyx_cm_density")
add_field("nyx_hi_density", units="nyx_prop_density")

dd["nyx_density"] / dd["nyx_hi_density"]    (should return dimensionless
YTArray)

- Casey

On Tue, May 7, 2013 at 8:18 AM, Matthew Turk <matthewturk at gmail.com> wrote:

> Hi Casey and Nathan,
>
> Last night when I read this I felt a bit of dread in my stomach,
> worrying that this would make things a lot more complex and difficult.
>  But I have to say, now having thought it through and read your
> pastes, reflected on the contents of the YTEP ... I think this is
> beautiful and elegant.  Very nicely thought out.
>
> I completely agree that -- particularly in 3.0 -- we no longer need to
> perform any conversions at IO time, and instead we should be pushing
> this to the places where the data will be presented to individual
> users.  This will require some thought about where we want this
> conversion to take place (i.e., at the display layer?) but it should
> be completely workable to do so.  The ensure_cgs decorator is a great
> way to manage this.
>
> I'm +1 on this, and can help with the technical details.  As it
> stands, all unit conversions happen in a single place in the code, so
> we can simply remove the items there and instead just wrap them in the
> appropriate "code" units.  We'll need to figure out the specific
> semantics of that but it should be okay.  How do we handle the fact
> that converting "code_density" to cgs is different for different
> arrays, depending on the code?  i.e., if I have two "code_density"
> arrays from a code that stores them in comoving code units, how do we
> distinguish between those two things during division, say?
>
> -Matt
>
> On Mon, May 6, 2013 at 6:50 PM, Casey W. Stark <caseywstark at gmail.com>
> wrote:
> > Hey Nathan.
> >
> > Thanks for the reminder about `ensure_cgs`. Definitely helps to see the
> > pseudocode too. Users could also write out the array.convert_to(...) in
> the
> > field function for whatever obscure units. I guess this is similar to the
> > fields with the units tacked on to the name, but much cleaner.
> >
> > I think making the code to universal field mapping more explicit will be
> > very helpful. I've been confused about where some derived fields come
> from
> > many times!
> >
> >
> > On Mon, May 6, 2013 at 3:39 PM, Nathan Goldbaum <goldbaum at ucolick.org>
> > wrote:
> >>
> >> Hey Casey,
> >>
> >> That's pretty much what I was thinking, although I wanted to add a
> little
> >> bit of syntactic sugar so the CGS conversion happens automatically:
> >>
> >> http://paste.yt-project.org/show/3443/
> >>
> >> The ensure_cgs decorator would simply call convert_to_cgs on the
> returned
> >> YTArray.  The nice thing about this is the flow of data is clear, things
> >> come in to the field definition code units and leave in CGS.  This also
> >> eliminates the need for the unit conversion functions that are scattered
> >> throughout the frontends and universal_fields.py.
> >>
> >> This has the nice feature that each of the code frontends will need to
> >> explicitly define mappings to the CGS fields (like Density) needed by
> >> universal fields.
> >>
> >> -Nathan
> >>
> >>
> >> On Mon, May 6, 2013 at 3:33 PM, Casey W. Stark <caseywstark at gmail.com>
> >> wrote:
> >>>
> >>> Hey yt.
> >>>
> >>> Nathan and I had a cool idea as an extension of the units work, but I
> am
> >>> hitting a yt internals knowledge wall writing a YTEP. I would like some
> >>> input on technical details or general feelings before I move ahead
> with the
> >>> write-up.
> >>>
> >>> One of the things mentioned in the units YTEP is handling code units.
> >>> Basically each frontend static output should register the code units.
> In Nyx
> >>> for instance, we would do something like registering "code_length" as
> Mpccm,
> >>> "code_mass" as Msun, etc. Then any field from the dataset can be
> loaded in
> >>> code units with something like dd["density"].in_units("code_density").
> >>>
> >>> This got us thinking -- why load native fields in CGS now? Instead, we
> >>> can define native fields exactly as they are on disk and users who
> want code
> >>> units will never have to convert by hand again. If frontend developers
> >>> define the fields and units on disk, and some mapping to the universal
> >>> fields, we shouldn't have to define or convert units anywhere else.
> >>>
> >>> I made a quick and dirty example of what we're thinking here:
> >>> http://paste.yt-project.org/show/3441/
> >>>
> >>> So, is something like this a good idea? Is it realistic? Would this
> make
> >>> for a simpler field info container? Nathan, is there anything I missed?
> >>>
> >>> - Casey
> >>>
> >>> _______________________________________________
> >>> yt-dev mailing list
> >>> yt-dev at lists.spacepope.org
> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>>
> >>
> >>
> >> _______________________________________________
> >> yt-dev mailing list
> >> yt-dev at lists.spacepope.org
> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>
> >
> >
> > _______________________________________________
> > yt-dev mailing list
> > yt-dev at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20130507/b456992b/attachment.htm>