[yt-dev] Native field discussion

Tue May 7 10:17:22 PDT 2013

> think of having two different parameter
files from the same simulation open at the same time.

Ah, got it. You're right that this will be a problem. The two datasets will
have separate unit registries, so something like "pccm" units can be
different from each. However, if you try something like
ds1_dd["enzo_comoving_density"] / ds2_dd["enzo_comoving_density"], sympy
might just cancel out the symbols in the unit expressions and the
conversion will not work. I'll try something...

On Tue, May 7, 2013 at 10:05 AM, Matthew Turk <matthewturk at gmail.com> wrote:

> On Tue, May 7, 2013 at 12:13 PM, Casey W. Stark <caseywstark at gmail.com>
> wrote:
> > Hey Matt.
> >
> > Excellent! It's a considerable change, so we should dread it a bit, but
> > really glad you like the idea too.
> >
> > About where the conversion should happen, I like the idea to return any
> > universal fields in cgs by default. Operations between "code" fields
> > ("mycode_density") would return in code units and ops between universal
> > fields ("density") would return in cgs. That means most users won't see a
> > change too. Mixing units is mostly straightforward. I think the only
> gotcha
> > is for add and sub, where we return in the left object's units.
>
> That sounds good to me.
>
> >
> >> How do we handle the fact
> > that converting "code_density" to cgs is different for different
> > arrays, depending on the code?  i.e., if I have two "code_density"
> > arrays from a code that stores them in comoving code units, how do we
> > distinguish between those two things during division, say?
> >
> > I'm not sure I'm following here, but is this a case where density field a
> > has different units from density field b? If so, I think we have to
> define
> > separate units for the code densities. Here's a quick Nyx example...
>
> Not exactly, but close -- think of having two different parameter
> files from the same simulation open at the same time.  If the sim
> stores internally in comoving units, the native->propercgs will vary
> between those two, but the registered units might be the same name,
> for instance "enzo_density" or something.
>
> >
> > unit_registry.add("nyx_cm_density", "Msun/Mpccm**3")
> > unit_registry.add("nyx_prop_density", "g/cm**3")
> >
> > add_field("nyx_density", units="nyx_cm_density")
> > add_field("nyx_hi_density", units="nyx_prop_density")
> >
> > dd["nyx_density"] / dd["nyx_hi_density"]    (should return dimensionless
> > YTArray)
> >
> > - Casey
> >
> >
> >
> > On Tue, May 7, 2013 at 8:18 AM, Matthew Turk <matthewturk at gmail.com>
> wrote:
> >>
> >> Hi Casey and Nathan,
> >>
> >> Last night when I read this I felt a bit of dread in my stomach,
> >> worrying that this would make things a lot more complex and difficult.
> >>  But I have to say, now having thought it through and read your
> >> pastes, reflected on the contents of the YTEP ... I think this is
> >> beautiful and elegant.  Very nicely thought out.
> >>
> >> I completely agree that -- particularly in 3.0 -- we no longer need to
> >> perform any conversions at IO time, and instead we should be pushing
> >> this to the places where the data will be presented to individual
> >> users.  This will require some thought about where we want this
> >> conversion to take place (i.e., at the display layer?) but it should
> >> be completely workable to do so.  The ensure_cgs decorator is a great
> >> way to manage this.
> >>
> >> I'm +1 on this, and can help with the technical details.  As it
> >> stands, all unit conversions happen in a single place in the code, so
> >> we can simply remove the items there and instead just wrap them in the
> >> appropriate "code" units.  We'll need to figure out the specific
> >> semantics of that but it should be okay.  How do we handle the fact
> >> that converting "code_density" to cgs is different for different
> >> arrays, depending on the code?  i.e., if I have two "code_density"
> >> arrays from a code that stores them in comoving code units, how do we
> >> distinguish between those two things during division, say?
> >>
> >> -Matt
> >>
> >> On Mon, May 6, 2013 at 6:50 PM, Casey W. Stark <caseywstark at gmail.com>
> >> wrote:
> >> > Hey Nathan.
> >> >
> >> > Thanks for the reminder about `ensure_cgs`. Definitely helps to see
> the
> >> > pseudocode too. Users could also write out the array.convert_to(...)
> in
> >> > the
> >> > field function for whatever obscure units. I guess this is similar to
> >> > the
> >> > fields with the units tacked on to the name, but much cleaner.
> >> >
> >> > I think making the code to universal field mapping more explicit will
> be
> >> > very helpful. I've been confused about where some derived fields come
> >> > from
> >> > many times!
> >> >
> >> >
> >> > On Mon, May 6, 2013 at 3:39 PM, Nathan Goldbaum <goldbaum at ucolick.org
> >
> >> > wrote:
> >> >>
> >> >> Hey Casey,
> >> >>
> >> >> That's pretty much what I was thinking, although I wanted to add a
> >> >> little
> >> >> bit of syntactic sugar so the CGS conversion happens automatically:
> >> >>
> >> >> http://paste.yt-project.org/show/3443/
> >> >>
> >> >> The ensure_cgs decorator would simply call convert_to_cgs on the
> >> >> returned
> >> >> YTArray.  The nice thing about this is the flow of data is clear,
> >> >> things
> >> >> come in to the field definition code units and leave in CGS.  This
> also
> >> >> eliminates the need for the unit conversion functions that are
> >> >> scattered
> >> >> throughout the frontends and universal_fields.py.
> >> >>
> >> >> This has the nice feature that each of the code frontends will need
> to
> >> >> explicitly define mappings to the CGS fields (like Density) needed by
> >> >> universal fields.
> >> >>
> >> >> -Nathan
> >> >>
> >> >>
> >> >> On Mon, May 6, 2013 at 3:33 PM, Casey W. Stark <
> caseywstark at gmail.com>
> >> >> wrote:
> >> >>>
> >> >>> Hey yt.
> >> >>>
> >> >>> Nathan and I had a cool idea as an extension of the units work, but
> I
> >> >>> am
> >> >>> hitting a yt internals knowledge wall writing a YTEP. I would like
> >> >>> some
> >> >>> input on technical details or general feelings before I move ahead
> >> >>> with the
> >> >>> write-up.
> >> >>>
> >> >>> One of the things mentioned in the units YTEP is handling code
> units.
> >> >>> Basically each frontend static output should register the code
> units.
> >> >>> In Nyx
> >> >>> for instance, we would do something like registering "code_length"
> as
> >> >>> Mpccm,
> >> >>> "code_mass" as Msun, etc. Then any field from the dataset can be
> >> >>> loaded in
> >> >>> code units with something like
> dd["density"].in_units("code_density").
> >> >>>
> >> >>> This got us thinking -- why load native fields in CGS now? Instead,
> we
> >> >>> can define native fields exactly as they are on disk and users who
> >> >>> want code
> >> >>> units will never have to convert by hand again. If frontend
> developers
> >> >>> define the fields and units on disk, and some mapping to the
> universal
> >> >>> fields, we shouldn't have to define or convert units anywhere else.
> >> >>>
> >> >>> I made a quick and dirty example of what we're thinking here:
> >> >>> http://paste.yt-project.org/show/3441/
> >> >>>
> >> >>> So, is something like this a good idea? Is it realistic? Would this
> >> >>> make
> >> >>> for a simpler field info container? Nathan, is there anything I
> >> >>> missed?
> >> >>>
> >> >>> - Casey
> >> >>>
> >> >>> _______________________________________________
> >> >>> yt-dev mailing list
> >> >>> yt-dev at lists.spacepope.org
> >> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>>
> >> >>
> >> >>
> >> >> _______________________________________________
> >> >> yt-dev mailing list
> >> >> yt-dev at lists.spacepope.org
> >> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > yt-dev mailing list
> >> > yt-dev at lists.spacepope.org
> >> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >
> >> _______________________________________________
> >> yt-dev mailing list
> >> yt-dev at lists.spacepope.org
> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >
> >
> >
> > _______________________________________________
> > yt-dev mailing list
> > yt-dev at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20130507/462ac725/attachment.html>