[yt-dev] Native field discussion

Nathan Goldbaum nathan12343 at gmail.com
Tue May 7 14:12:41 PDT 2013


Definitely true.  Perhaps a simulation could register its self as dimensionless in the same way cosmological simulations are registered so that the unit registry is constructed appropriately at load time.

Instead of @ensure_cgs, we could have something like @ensure_universal_units which will make sure the universal fields have the appropriate units for that context.  This does add some additional complexity and I'm not sure how hard it would be to bend the unit system to do the right thing.

On May 7, 2013, at 2:06 PM, j s oishi <jsoishi at gmail.com> wrote:

> 
> 
> 
> On Tue, May 7, 2013 at 1:25 PM, Matthew Turk <matthewturk at gmail.com> wrote:
> On Tue, May 7, 2013 at 1:24 PM, Casey W. Stark <caseywstark at gmail.com> wrote:
> > Ok, I just tried this and it doesn't work. Even though the Unit objects are
> > separate instances, the sympy Symbol objects in the expressions are treated
> > as the same. We will have to think about the design some more.
> 
> Although, on reflection, maybe *this is okay*.  We should perhaps
> assume that anyone who is actually dividing or multiplying or
> something the native code units knows what they're doing.  If they
> want to look at the cgs units, they should be doing so.  I'm not
> entirely sure.
> 
> I'm not so sure about this. What about simulations that are performed in dimensionless units? This is something that yt handles poorly right now, and I think this system should fix that. For example, in a dimensionless system, the code units are fully functional and probably what you want to work in. Or am I missing something here?
>  
> 
> >
> >
> > On Tue, May 7, 2013 at 10:17 AM, Casey W. Stark <caseywstark at gmail.com>
> > wrote:
> >>
> >> > think of having two different parameter
> >> files from the same simulation open at the same time.
> >>
> >> Ah, got it. You're right that this will be a problem. The two datasets
> >> will have separate unit registries, so something like "pccm" units can be
> >> different from each. However, if you try something like
> >> ds1_dd["enzo_comoving_density"] / ds2_dd["enzo_comoving_density"], sympy
> >> might just cancel out the symbols in the unit expressions and the conversion
> >> will not work. I'll try something...
> >>
> >>
> >> On Tue, May 7, 2013 at 10:05 AM, Matthew Turk <matthewturk at gmail.com>
> >> wrote:
> >>>
> >>> On Tue, May 7, 2013 at 12:13 PM, Casey W. Stark <caseywstark at gmail.com>
> >>> wrote:
> >>> > Hey Matt.
> >>> >
> >>> > Excellent! It's a considerable change, so we should dread it a bit, but
> >>> > really glad you like the idea too.
> >>> >
> >>> > About where the conversion should happen, I like the idea to return any
> >>> > universal fields in cgs by default. Operations between "code" fields
> >>> > ("mycode_density") would return in code units and ops between universal
> >>> > fields ("density") would return in cgs. That means most users won't see
> >>> > a
> >>> > change too. Mixing units is mostly straightforward. I think the only
> >>> > gotcha
> >>> > is for add and sub, where we return in the left object's units.
> >>>
> >>> That sounds good to me.
> >>>
> >>> >
> >>> >> How do we handle the fact
> >>> > that converting "code_density" to cgs is different for different
> >>> > arrays, depending on the code?  i.e., if I have two "code_density"
> >>> > arrays from a code that stores them in comoving code units, how do we
> >>> > distinguish between those two things during division, say?
> >>> >
> >>> > I'm not sure I'm following here, but is this a case where density field
> >>> > a
> >>> > has different units from density field b? If so, I think we have to
> >>> > define
> >>> > separate units for the code densities. Here's a quick Nyx example...
> >>>
> >>> Not exactly, but close -- think of having two different parameter
> >>> files from the same simulation open at the same time.  If the sim
> >>> stores internally in comoving units, the native->propercgs will vary
> >>> between those two, but the registered units might be the same name,
> >>> for instance "enzo_density" or something.
> >>>
> >>> >
> >>> > unit_registry.add("nyx_cm_density", "Msun/Mpccm**3")
> >>> > unit_registry.add("nyx_prop_density", "g/cm**3")
> >>> >
> >>> > add_field("nyx_density", units="nyx_cm_density")
> >>> > add_field("nyx_hi_density", units="nyx_prop_density")
> >>> >
> >>> > dd["nyx_density"] / dd["nyx_hi_density"]    (should return
> >>> > dimensionless
> >>> > YTArray)
> >>> >
> >>> > - Casey
> >>> >
> >>> >
> >>> >
> >>> > On Tue, May 7, 2013 at 8:18 AM, Matthew Turk <matthewturk at gmail.com>
> >>> > wrote:
> >>> >>
> >>> >> Hi Casey and Nathan,
> >>> >>
> >>> >> Last night when I read this I felt a bit of dread in my stomach,
> >>> >> worrying that this would make things a lot more complex and difficult.
> >>> >>  But I have to say, now having thought it through and read your
> >>> >> pastes, reflected on the contents of the YTEP ... I think this is
> >>> >> beautiful and elegant.  Very nicely thought out.
> >>> >>
> >>> >> I completely agree that -- particularly in 3.0 -- we no longer need to
> >>> >> perform any conversions at IO time, and instead we should be pushing
> >>> >> this to the places where the data will be presented to individual
> >>> >> users.  This will require some thought about where we want this
> >>> >> conversion to take place (i.e., at the display layer?) but it should
> >>> >> be completely workable to do so.  The ensure_cgs decorator is a great
> >>> >> way to manage this.
> >>> >>
> >>> >> I'm +1 on this, and can help with the technical details.  As it
> >>> >> stands, all unit conversions happen in a single place in the code, so
> >>> >> we can simply remove the items there and instead just wrap them in the
> >>> >> appropriate "code" units.  We'll need to figure out the specific
> >>> >> semantics of that but it should be okay.  How do we handle the fact
> >>> >> that converting "code_density" to cgs is different for different
> >>> >> arrays, depending on the code?  i.e., if I have two "code_density"
> >>> >> arrays from a code that stores them in comoving code units, how do we
> >>> >> distinguish between those two things during division, say?
> >>> >>
> >>> >> -Matt
> >>> >>
> >>> >> On Mon, May 6, 2013 at 6:50 PM, Casey W. Stark <caseywstark at gmail.com>
> >>> >> wrote:
> >>> >> > Hey Nathan.
> >>> >> >
> >>> >> > Thanks for the reminder about `ensure_cgs`. Definitely helps to see
> >>> >> > the
> >>> >> > pseudocode too. Users could also write out the array.convert_to(...)
> >>> >> > in
> >>> >> > the
> >>> >> > field function for whatever obscure units. I guess this is similar
> >>> >> > to
> >>> >> > the
> >>> >> > fields with the units tacked on to the name, but much cleaner.
> >>> >> >
> >>> >> > I think making the code to universal field mapping more explicit
> >>> >> > will be
> >>> >> > very helpful. I've been confused about where some derived fields
> >>> >> > come
> >>> >> > from
> >>> >> > many times!
> >>> >> >
> >>> >> >
> >>> >> > On Mon, May 6, 2013 at 3:39 PM, Nathan Goldbaum
> >>> >> > <goldbaum at ucolick.org>
> >>> >> > wrote:
> >>> >> >>
> >>> >> >> Hey Casey,
> >>> >> >>
> >>> >> >> That's pretty much what I was thinking, although I wanted to add a
> >>> >> >> little
> >>> >> >> bit of syntactic sugar so the CGS conversion happens automatically:
> >>> >> >>
> >>> >> >> http://paste.yt-project.org/show/3443/
> >>> >> >>
> >>> >> >> The ensure_cgs decorator would simply call convert_to_cgs on the
> >>> >> >> returned
> >>> >> >> YTArray.  The nice thing about this is the flow of data is clear,
> >>> >> >> things
> >>> >> >> come in to the field definition code units and leave in CGS.  This
> >>> >> >> also
> >>> >> >> eliminates the need for the unit conversion functions that are
> >>> >> >> scattered
> >>> >> >> throughout the frontends and universal_fields.py.
> >>> >> >>
> >>> >> >> This has the nice feature that each of the code frontends will need
> >>> >> >> to
> >>> >> >> explicitly define mappings to the CGS fields (like Density) needed
> >>> >> >> by
> >>> >> >> universal fields.
> >>> >> >>
> >>> >> >> -Nathan
> >>> >> >>
> >>> >> >>
> >>> >> >> On Mon, May 6, 2013 at 3:33 PM, Casey W. Stark
> >>> >> >> <caseywstark at gmail.com>
> >>> >> >> wrote:
> >>> >> >>>
> >>> >> >>> Hey yt.
> >>> >> >>>
> >>> >> >>> Nathan and I had a cool idea as an extension of the units work,
> >>> >> >>> but I
> >>> >> >>> am
> >>> >> >>> hitting a yt internals knowledge wall writing a YTEP. I would like
> >>> >> >>> some
> >>> >> >>> input on technical details or general feelings before I move ahead
> >>> >> >>> with the
> >>> >> >>> write-up.
> >>> >> >>>
> >>> >> >>> One of the things mentioned in the units YTEP is handling code
> >>> >> >>> units.
> >>> >> >>> Basically each frontend static output should register the code
> >>> >> >>> units.
> >>> >> >>> In Nyx
> >>> >> >>> for instance, we would do something like registering "code_length"
> >>> >> >>> as
> >>> >> >>> Mpccm,
> >>> >> >>> "code_mass" as Msun, etc. Then any field from the dataset can be
> >>> >> >>> loaded in
> >>> >> >>> code units with something like
> >>> >> >>> dd["density"].in_units("code_density").
> >>> >> >>>
> >>> >> >>> This got us thinking -- why load native fields in CGS now?
> >>> >> >>> Instead, we
> >>> >> >>> can define native fields exactly as they are on disk and users who
> >>> >> >>> want code
> >>> >> >>> units will never have to convert by hand again. If frontend
> >>> >> >>> developers
> >>> >> >>> define the fields and units on disk, and some mapping to the
> >>> >> >>> universal
> >>> >> >>> fields, we shouldn't have to define or convert units anywhere
> >>> >> >>> else.
> >>> >> >>>
> >>> >> >>> I made a quick and dirty example of what we're thinking here:
> >>> >> >>> http://paste.yt-project.org/show/3441/
> >>> >> >>>
> >>> >> >>> So, is something like this a good idea? Is it realistic? Would
> >>> >> >>> this
> >>> >> >>> make
> >>> >> >>> for a simpler field info container? Nathan, is there anything I
> >>> >> >>> missed?
> >>> >> >>>
> >>> >> >>> - Casey
> >>> >> >>>
> >>> >> >>> _______________________________________________
> >>> >> >>> yt-dev mailing list
> >>> >> >>> yt-dev at lists.spacepope.org
> >>> >> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>>
> >>> >> >>
> >>> >> >>
> >>> >> >> _______________________________________________
> >>> >> >> yt-dev mailing list
> >>> >> >> yt-dev at lists.spacepope.org
> >>> >> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >>
> >>> >> >
> >>> >> >
> >>> >> > _______________________________________________
> >>> >> > yt-dev mailing list
> >>> >> > yt-dev at lists.spacepope.org
> >>> >> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >> >
> >>> >> _______________________________________________
> >>> >> yt-dev mailing list
> >>> >> yt-dev at lists.spacepope.org
> >>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >
> >>> >
> >>> >
> >>> > _______________________________________________
> >>> > yt-dev mailing list
> >>> > yt-dev at lists.spacepope.org
> >>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>> >
> >>> _______________________________________________
> >>> yt-dev mailing list
> >>> yt-dev at lists.spacepope.org
> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>
> >>
> >
> >
> > _______________________________________________
> > yt-dev mailing list
> > yt-dev at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> 
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20130507/d80af2e3/attachment.html>


More information about the yt-dev mailing list