[yt-dev] Native field discussion

j s oishi jsoishi at gmail.com
Tue May 7 14:06:31 PDT 2013 

On Tue, May 7, 2013 at 1:25 PM, Matthew Turk <matthewturk at gmail.com> wrote:

> On Tue, May 7, 2013 at 1:24 PM, Casey W. Stark <caseywstark at gmail.com>
> wrote:
> > Ok, I just tried this and it doesn't work. Even though the Unit objects
> are
> > separate instances, the sympy Symbol objects in the expressions are
> treated
> > as the same. We will have to think about the design some more.
>
> Although, on reflection, maybe *this is okay*.  We should perhaps
> assume that anyone who is actually dividing or multiplying or
> something the native code units knows what they're doing.  If they
> want to look at the cgs units, they should be doing so.  I'm not
> entirely sure.
>

I'm not so sure about this. What about simulations that are performed in
dimensionless units? This is something that yt handles poorly right now,
and I think this system should fix that. For example, in a dimensionless
system, the code units are fully functional and probably what you want to
work in. Or am I missing something here?


>
> >
> >
> > On Tue, May 7, 2013 at 10:17 AM, Casey W. Stark <caseywstark at gmail.com>
> > wrote:
> >>
> >> > think of having two different parameter
> >> files from the same simulation open at the same time.
> >>
> >> Ah, got it. You're right that this will be a problem. The two datasets
> >> will have separate unit registries, so something like "pccm" units can
> be
> >> different from each. However, if you try something like
> >> ds1_dd["enzo_comoving_density"] / ds2_dd["enzo_comoving_density"], sympy
> >> might just cancel out the symbols in the unit expressions and the
> conversion
> >> will not work. I'll try something...
> >>
> >>
> >> On Tue, May 7, 2013 at 10:05 AM, Matthew Turk <matthewturk at gmail.com>
> >> wrote:
> >>>
> >>> On Tue, May 7, 2013 at 12:13 PM, Casey W. Stark <caseywstark at gmail.com
> >
> >>> wrote:
> >>> > Hey Matt.
> >>> >
> >>> > Excellent! It's a considerable change, so we should dread it a bit,
> but
> >>> > really glad you like the idea too.
> >>> >
> >>> > About where the conversion should happen, I like the idea to return
> any
> >>> > universal fields in cgs by default. Operations between "code" fields
> >>> > ("mycode_density") would return in code units and ops between
> universal
> >>> > fields ("density") would return in cgs. That means most users won't
> see
> >>> > a
> >>> > change too. Mixing units is mostly straightforward. I think the only
> >>> > gotcha
> >>> > is for add and sub, where we return in the left object's units.
> >>>
> >>> That sounds good to me.
> >>>
> >>> >
> >>> >> How do we handle the fact
> >>> > that converting "code_density" to cgs is different for different
> >>> > arrays, depending on the code?  i.e., if I have two "code_density"
> >>> > arrays from a code that stores them in comoving code units, how do we
> >>> > distinguish between those two things during division, say?
> >>> >
> >>> > I'm not sure I'm following here, but is this a case where density
> field
> >>> > a
> >>> > has different units from density field b? If so, I think we have to
> >>> > define
> >>> > separate units for the code densities. Here's a quick Nyx example...
> >>>
> >>> Not exactly, but close -- think of having two different parameter
> >>> files from the same simulation open at the same time.  If the sim
> >>> stores internally in comoving units, the native->propercgs will vary
> >>> between those two, but the registered units might be the same name,
> >>> for instance "enzo_density" or something.
> >>>
> >>> >
> >>> > unit_registry.add("nyx_cm_density", "Msun/Mpccm**3")
> >>> > unit_registry.add("nyx_prop_density", "g/cm**3")
> >>> >
> >>> > add_field("nyx_density", units="nyx_cm_density")
> >>> > add_field("nyx_hi_density", units="nyx_prop_density")
> >>> >
> >>> > dd["nyx_density"] / dd["nyx_hi_density"]    (should return
> >>> > dimensionless
> >>> > YTArray)
> >>> >
> >>> > - Casey
> >>> >
> >>> >
> >>> >
> >>> > On Tue, May 7, 2013 at 8:18 AM, Matthew Turk <matthewturk at gmail.com>
> >>> > wrote:
> >>> >>
> >>> >> Hi Casey and Nathan,
> >>> >>
> >>> >> Last night when I read this I felt a bit of dread in my stomach,
> >>> >> worrying that this would make things a lot more complex and
> difficult.
> >>> >>  But I have to say, now having thought it through and read your
> >>> >> pastes, reflected on the contents of the YTEP ... I think this is
> >>> >> beautiful and elegant.  Very nicely thought out.
> >>> >>
> >>> >> I completely agree that -- particularly in 3.0 -- we no longer need
> to
> >>> >> perform any conversions at IO time, and instead we should be pushing
> >>> >> this to the places where the data will be presented to individual
> >>> >> users.  This will require some thought about where we want this
> >>> >> conversion to take place (i.e., at the display layer?) but it should
> >>> >> be completely workable to do so.  The ensure_cgs decorator is a
> great
> >>> >> way to manage this.
> >>> >>
> >>> >> I'm +1 on this, and can help with the technical details.  As it
> >>> >> stands, all unit conversions happen in a single place in the code,
> so
> >>> >> we can simply remove the items there and instead just wrap them in
> the
> >>> >> appropriate "code" units.  We'll need to figure out the specific
> >>> >> semantics of that but it should be okay.  How do we handle the fact
> >>> >> that converting "code_density" to cgs is different for different
> >>> >> arrays, depending on the code?  i.e., if I have two "code_density"
> >>> >> arrays from a code that stores them in comoving code units, how do
> we
> >>> >> distinguish between those two things during division, say?
> >>> >>
> >>> >> -Matt
> >>> >>
> >>> >> On Mon, May 6, 2013 at 6:50 PM, Casey W. Stark <
> caseywstark at gmail.com>
> >>> >> wrote:
> >>> >> > Hey Nathan.
> >>> >> >
> >>> >> > Thanks for the reminder about `ensure_cgs`. Definitely helps to
> see
> >>> >> > the
> >>> >> > pseudocode too. Users could also write out the
> array.convert_to(...)
> >>> >> > in
> >>> >> > the
> >>> >> > field function for whatever obscure units. I guess this is similar
> >>> >> > to
> >>> >> > the
> >>> >> > fields with the units tacked on to the name, but much cleaner.
> >>> >> >
> >>> >> > I think making the code to universal field mapping more explicit
> >>> >> > will be
> >>> >> > very helpful. I've been confused about where some derived fields
> >>> >> > come
> >>> >> > from
> >>> >> > many times!
> >>> >> >
> >>> >> >
> >>> >> > On Mon, May 6, 2013 at 3:39 PM, Nathan Goldbaum
> >>> >> > <goldbaum at ucolick.org>
> >>> >> > wrote:
> >>> >> >>
> >>> >> >> Hey Casey,
> >>> >> >>
> >>> >> >> That's pretty much what I was thinking, although I wanted to add
> a
> >>> >> >> little
> >>> >> >> bit of syntactic sugar so the CGS conversion happens
> automatically:
> >>> >> >>
> >>> >> >> http://paste.yt-project.org/show/3443/
> >>> >> >>
> >>> >> >> The ensure_cgs decorator would simply call convert_to_cgs on the
> >>> >> >> returned
> >>> >> >> YTArray.  The nice thing about this is the flow of data is clear,
> >>> >> >> things
> >>> >> >> come in to the field definition code units and leave in CGS.
>  This
> >>> >> >> also
> >>> >> >> eliminates the need for the unit conversion functions that are
> >>> >> >> scattered
> >>> >> >> throughout the frontends and universal_fields.py.
> >>> >> >>
> >>> >> >> This has the nice feature that each of the code frontends will
> need
> >>> >> >> to
> >>> >> >> explicitly define mappings to the CGS fields (like Density)
> needed
> >>> >> >> by
> >>> >> >> universal fields.
> >>> >> >>
> >>> >> >> -Nathan
> >>> >> >>
> >>> >> >>
> >>> >> >> On Mon, May 6, 2013 at 3:33 PM, Casey W. Stark
> >>> >> >> <caseywstark at gmail.com>
> >>> >> >> wrote:
> >>> >> >>>
> >>> >> >>> Hey yt.
> >>> >> >>>
> >>> >> >>> Nathan and I had a cool idea as an extension of the units work,
> >>> >> >>> but I
> >>> >> >>> am
> >>> >> >>> hitting a yt internals knowledge wall writing a YTEP. I would
> like
> >>> >> >>> some
> >>> >> >>> input on technical details or general feelings before I move
> ahead
> >>> >> >>> with the
> >>> >> >>> write-up.
> >>> >> >>>
> >>> >> >>> One of the things mentioned in the units YTEP is handling code
> >>> >> >>> units.
> >>> >> >>> Basically each frontend static output should register the code
> >>> >> >>> units.
> >>> >> >>> In Nyx
> >>> >> >>> for instance, we would do something like registering
> "code_length"
> >>> >> >>> as
> >>> >> >>> Mpccm,
> >>> >> >>> "code_mass" as Msun, etc. Then any field from the dataset can be
> >>> >> >>> loaded in
> >>> >> >>> code units with something like
> >>> >> >>> dd["density"].in_units("code_density").
> >>> >> >>>
> >>> >> >>> This got us thinking -- why load native fields in CGS now?
> >>> >> >>> Instead, we
> >>> >> >>> can define native fields exactly as they are on disk and users
> who
> >>> >> >>> want code
> >>> >> >>> units will never have to convert by hand again. If frontend
> >>> >> >>> developers
> >>> >> >>> define the fields and units on disk, and some mapping to the
> >>> >> >>> universal
> >>> >> >>> fields, we shouldn't have to define or convert units anywhere
> >>> >> >>> else.
> >>> >> >>>
> >>> >> >>> I made a quick and dirty example of what we're thinking here:
> >>> >> >>> http://paste.yt-project.org/show/3441/
> >>> >> >>>
> >>> >> >>> So, is something like this a good idea? Is it realistic? Would
> >>> >> >>> this
> >>> >> >>> make
> >>> >> >>> for a simpler field info container? Nathan, is there anything I
> >>> >> >>> missed?
> >>> >> >>>
> >>> >> >>> - Casey
> >>> >> >>>
> >>> >> >>> _______________________________________________
> >>> >> >>> yt-dev mailing list
> >>> >> >>> yt-dev at lists.spacepope.org
> >>> >> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>>
> >>> >> >>
> >>> >> >>
> >>> >> >> _______________________________________________
> >>> >> >> yt-dev mailing list
> >>> >> >> yt-dev at lists.spacepope.org
> >>> >> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>
> >>> >> >
> >>> >> >
> >>> >> > _______________________________________________
> >>> >> > yt-dev mailing list
> >>> >> > yt-dev at lists.spacepope.org
> >>> >> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >
> >>> >> _______________________________________________
> >>> >> yt-dev mailing list
> >>> >> yt-dev at lists.spacepope.org
> >>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >
> >>> >
> >>> >
> >>> > _______________________________________________
> >>> > yt-dev mailing list
> >>> > yt-dev at lists.spacepope.org
> >>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >
> >>> _______________________________________________
> >>> yt-dev mailing list
> >>> yt-dev at lists.spacepope.org
> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>
> >>
> >
> >
> > _______________________________________________
> > yt-dev mailing list
> > yt-dev at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20130507/167072c4/attachment.htm>

More information about the yt-dev mailing list