[yt-dev] Native field discussion

Casey W. Stark caseywstark at gmail.com
Tue May 7 10:24:37 PDT 2013


Ok, I just tried this and it doesn't work. Even though the Unit objects are
separate instances, the sympy Symbol objects in the expressions are treated
as the same. We will have to think about the design some more.


On Tue, May 7, 2013 at 10:17 AM, Casey W. Stark <caseywstark at gmail.com>wrote:

> > think of having two different parameter
> files from the same simulation open at the same time.
>
> Ah, got it. You're right that this will be a problem. The two datasets
> will have separate unit registries, so something like "pccm" units can be
> different from each. However, if you try something like
> ds1_dd["enzo_comoving_density"] / ds2_dd["enzo_comoving_density"], sympy
> might just cancel out the symbols in the unit expressions and the
> conversion will not work. I'll try something...
>
>
> On Tue, May 7, 2013 at 10:05 AM, Matthew Turk <matthewturk at gmail.com>wrote:
>
>> On Tue, May 7, 2013 at 12:13 PM, Casey W. Stark <caseywstark at gmail.com>
>> wrote:
>> > Hey Matt.
>> >
>> > Excellent! It's a considerable change, so we should dread it a bit, but
>> > really glad you like the idea too.
>> >
>> > About where the conversion should happen, I like the idea to return any
>> > universal fields in cgs by default. Operations between "code" fields
>> > ("mycode_density") would return in code units and ops between universal
>> > fields ("density") would return in cgs. That means most users won't see
>> a
>> > change too. Mixing units is mostly straightforward. I think the only
>> gotcha
>> > is for add and sub, where we return in the left object's units.
>>
>> That sounds good to me.
>>
>> >
>> >> How do we handle the fact
>> > that converting "code_density" to cgs is different for different
>> > arrays, depending on the code?  i.e., if I have two "code_density"
>> > arrays from a code that stores them in comoving code units, how do we
>> > distinguish between those two things during division, say?
>> >
>> > I'm not sure I'm following here, but is this a case where density field
>> a
>> > has different units from density field b? If so, I think we have to
>> define
>> > separate units for the code densities. Here's a quick Nyx example...
>>
>> Not exactly, but close -- think of having two different parameter
>> files from the same simulation open at the same time.  If the sim
>> stores internally in comoving units, the native->propercgs will vary
>> between those two, but the registered units might be the same name,
>> for instance "enzo_density" or something.
>>
>> >
>> > unit_registry.add("nyx_cm_density", "Msun/Mpccm**3")
>> > unit_registry.add("nyx_prop_density", "g/cm**3")
>> >
>> > add_field("nyx_density", units="nyx_cm_density")
>> > add_field("nyx_hi_density", units="nyx_prop_density")
>> >
>> > dd["nyx_density"] / dd["nyx_hi_density"]    (should return dimensionless
>> > YTArray)
>> >
>> > - Casey
>> >
>> >
>> >
>> > On Tue, May 7, 2013 at 8:18 AM, Matthew Turk <matthewturk at gmail.com>
>> wrote:
>> >>
>> >> Hi Casey and Nathan,
>> >>
>> >> Last night when I read this I felt a bit of dread in my stomach,
>> >> worrying that this would make things a lot more complex and difficult.
>> >>  But I have to say, now having thought it through and read your
>> >> pastes, reflected on the contents of the YTEP ... I think this is
>> >> beautiful and elegant.  Very nicely thought out.
>> >>
>> >> I completely agree that -- particularly in 3.0 -- we no longer need to
>> >> perform any conversions at IO time, and instead we should be pushing
>> >> this to the places where the data will be presented to individual
>> >> users.  This will require some thought about where we want this
>> >> conversion to take place (i.e., at the display layer?) but it should
>> >> be completely workable to do so.  The ensure_cgs decorator is a great
>> >> way to manage this.
>> >>
>> >> I'm +1 on this, and can help with the technical details.  As it
>> >> stands, all unit conversions happen in a single place in the code, so
>> >> we can simply remove the items there and instead just wrap them in the
>> >> appropriate "code" units.  We'll need to figure out the specific
>> >> semantics of that but it should be okay.  How do we handle the fact
>> >> that converting "code_density" to cgs is different for different
>> >> arrays, depending on the code?  i.e., if I have two "code_density"
>> >> arrays from a code that stores them in comoving code units, how do we
>> >> distinguish between those two things during division, say?
>> >>
>> >> -Matt
>> >>
>> >> On Mon, May 6, 2013 at 6:50 PM, Casey W. Stark <caseywstark at gmail.com>
>> >> wrote:
>> >> > Hey Nathan.
>> >> >
>> >> > Thanks for the reminder about `ensure_cgs`. Definitely helps to see
>> the
>> >> > pseudocode too. Users could also write out the array.convert_to(...)
>> in
>> >> > the
>> >> > field function for whatever obscure units. I guess this is similar to
>> >> > the
>> >> > fields with the units tacked on to the name, but much cleaner.
>> >> >
>> >> > I think making the code to universal field mapping more explicit
>> will be
>> >> > very helpful. I've been confused about where some derived fields come
>> >> > from
>> >> > many times!
>> >> >
>> >> >
>> >> > On Mon, May 6, 2013 at 3:39 PM, Nathan Goldbaum <
>> goldbaum at ucolick.org>
>> >> > wrote:
>> >> >>
>> >> >> Hey Casey,
>> >> >>
>> >> >> That's pretty much what I was thinking, although I wanted to add a
>> >> >> little
>> >> >> bit of syntactic sugar so the CGS conversion happens automatically:
>> >> >>
>> >> >> http://paste.yt-project.org/show/3443/
>> >> >>
>> >> >> The ensure_cgs decorator would simply call convert_to_cgs on the
>> >> >> returned
>> >> >> YTArray.  The nice thing about this is the flow of data is clear,
>> >> >> things
>> >> >> come in to the field definition code units and leave in CGS.  This
>> also
>> >> >> eliminates the need for the unit conversion functions that are
>> >> >> scattered
>> >> >> throughout the frontends and universal_fields.py.
>> >> >>
>> >> >> This has the nice feature that each of the code frontends will need
>> to
>> >> >> explicitly define mappings to the CGS fields (like Density) needed
>> by
>> >> >> universal fields.
>> >> >>
>> >> >> -Nathan
>> >> >>
>> >> >>
>> >> >> On Mon, May 6, 2013 at 3:33 PM, Casey W. Stark <
>> caseywstark at gmail.com>
>> >> >> wrote:
>> >> >>>
>> >> >>> Hey yt.
>> >> >>>
>> >> >>> Nathan and I had a cool idea as an extension of the units work,
>> but I
>> >> >>> am
>> >> >>> hitting a yt internals knowledge wall writing a YTEP. I would like
>> >> >>> some
>> >> >>> input on technical details or general feelings before I move ahead
>> >> >>> with the
>> >> >>> write-up.
>> >> >>>
>> >> >>> One of the things mentioned in the units YTEP is handling code
>> units.
>> >> >>> Basically each frontend static output should register the code
>> units.
>> >> >>> In Nyx
>> >> >>> for instance, we would do something like registering "code_length"
>> as
>> >> >>> Mpccm,
>> >> >>> "code_mass" as Msun, etc. Then any field from the dataset can be
>> >> >>> loaded in
>> >> >>> code units with something like
>> dd["density"].in_units("code_density").
>> >> >>>
>> >> >>> This got us thinking -- why load native fields in CGS now?
>> Instead, we
>> >> >>> can define native fields exactly as they are on disk and users who
>> >> >>> want code
>> >> >>> units will never have to convert by hand again. If frontend
>> developers
>> >> >>> define the fields and units on disk, and some mapping to the
>> universal
>> >> >>> fields, we shouldn't have to define or convert units anywhere else.
>> >> >>>
>> >> >>> I made a quick and dirty example of what we're thinking here:
>> >> >>> http://paste.yt-project.org/show/3441/
>> >> >>>
>> >> >>> So, is something like this a good idea? Is it realistic? Would this
>> >> >>> make
>> >> >>> for a simpler field info container? Nathan, is there anything I
>> >> >>> missed?
>> >> >>>
>> >> >>> - Casey
>> >> >>>
>> >> >>> _______________________________________________
>> >> >>> yt-dev mailing list
>> >> >>> yt-dev at lists.spacepope.org
>> >> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >> >>>
>> >> >>
>> >> >>
>> >> >> _______________________________________________
>> >> >> yt-dev mailing list
>> >> >> yt-dev at lists.spacepope.org
>> >> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >> >>
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > yt-dev mailing list
>> >> > yt-dev at lists.spacepope.org
>> >> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >> >
>> >> _______________________________________________
>> >> yt-dev mailing list
>> >> yt-dev at lists.spacepope.org
>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> >
>> >
>> > _______________________________________________
>> > yt-dev mailing list
>> > yt-dev at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20130507/ab1e6cb8/attachment.htm>


More information about the yt-dev mailing list