[yt-dev] Native field discussion

Greg Bryan gbryan04 at gmail.com
Tue May 7 13:40:52 PDT 2013 

I'm not sure about the general case, but if I divided two comoving density fields, I would certainly not expect (or want) any conversion.

Greg

p.s. I really like the unit work.

On May 7, 2013, at 1:25 PM, Matthew Turk <matthewturk at gmail.com> wrote:

> On Tue, May 7, 2013 at 1:24 PM, Casey W. Stark <caseywstark at gmail.com> wrote:
>> Ok, I just tried this and it doesn't work. Even though the Unit objects are
>> separate instances, the sympy Symbol objects in the expressions are treated
>> as the same. We will have to think about the design some more.
> 
> Although, on reflection, maybe *this is okay*.  We should perhaps
> assume that anyone who is actually dividing or multiplying or
> something the native code units knows what they're doing.  If they
> want to look at the cgs units, they should be doing so.  I'm not
> entirely sure.
> 
>> 
>> 
>> On Tue, May 7, 2013 at 10:17 AM, Casey W. Stark <caseywstark at gmail.com>
>> wrote:
>>> 
>>>> think of having two different parameter
>>> files from the same simulation open at the same time.
>>> 
>>> Ah, got it. You're right that this will be a problem. The two datasets
>>> will have separate unit registries, so something like "pccm" units can be
>>> different from each. However, if you try something like
>>> ds1_dd["enzo_comoving_density"] / ds2_dd["enzo_comoving_density"], sympy
>>> might just cancel out the symbols in the unit expressions and the conversion
>>> will not work. I'll try something...
>>> 
>>> 
>>> On Tue, May 7, 2013 at 10:05 AM, Matthew Turk <matthewturk at gmail.com>
>>> wrote:
>>>> 
>>>> On Tue, May 7, 2013 at 12:13 PM, Casey W. Stark <caseywstark at gmail.com>
>>>> wrote:
>>>>> Hey Matt.
>>>>> 
>>>>> Excellent! It's a considerable change, so we should dread it a bit, but
>>>>> really glad you like the idea too.
>>>>> 
>>>>> About where the conversion should happen, I like the idea to return any
>>>>> universal fields in cgs by default. Operations between "code" fields
>>>>> ("mycode_density") would return in code units and ops between universal
>>>>> fields ("density") would return in cgs. That means most users won't see
>>>>> a
>>>>> change too. Mixing units is mostly straightforward. I think the only
>>>>> gotcha
>>>>> is for add and sub, where we return in the left object's units.
>>>> 
>>>> That sounds good to me.
>>>> 
>>>>> 
>>>>>> How do we handle the fact
>>>>> that converting "code_density" to cgs is different for different
>>>>> arrays, depending on the code?  i.e., if I have two "code_density"
>>>>> arrays from a code that stores them in comoving code units, how do we
>>>>> distinguish between those two things during division, say?
>>>>> 
>>>>> I'm not sure I'm following here, but is this a case where density field
>>>>> a
>>>>> has different units from density field b? If so, I think we have to
>>>>> define
>>>>> separate units for the code densities. Here's a quick Nyx example...
>>>> 
>>>> Not exactly, but close -- think of having two different parameter
>>>> files from the same simulation open at the same time.  If the sim
>>>> stores internally in comoving units, the native->propercgs will vary
>>>> between those two, but the registered units might be the same name,
>>>> for instance "enzo_density" or something.
>>>> 
>>>>> 
>>>>> unit_registry.add("nyx_cm_density", "Msun/Mpccm**3")
>>>>> unit_registry.add("nyx_prop_density", "g/cm**3")
>>>>> 
>>>>> add_field("nyx_density", units="nyx_cm_density")
>>>>> add_field("nyx_hi_density", units="nyx_prop_density")
>>>>> 
>>>>> dd["nyx_density"] / dd["nyx_hi_density"]    (should return
>>>>> dimensionless
>>>>> YTArray)
>>>>> 
>>>>> - Casey
>>>>> 
>>>>> 
>>>>> 
>>>>> On Tue, May 7, 2013 at 8:18 AM, Matthew Turk <matthewturk at gmail.com>
>>>>> wrote:
>>>>>> 
>>>>>> Hi Casey and Nathan,
>>>>>> 
>>>>>> Last night when I read this I felt a bit of dread in my stomach,
>>>>>> worrying that this would make things a lot more complex and difficult.
>>>>>> But I have to say, now having thought it through and read your
>>>>>> pastes, reflected on the contents of the YTEP ... I think this is
>>>>>> beautiful and elegant.  Very nicely thought out.
>>>>>> 
>>>>>> I completely agree that -- particularly in 3.0 -- we no longer need to
>>>>>> perform any conversions at IO time, and instead we should be pushing
>>>>>> this to the places where the data will be presented to individual
>>>>>> users.  This will require some thought about where we want this
>>>>>> conversion to take place (i.e., at the display layer?) but it should
>>>>>> be completely workable to do so.  The ensure_cgs decorator is a great
>>>>>> way to manage this.
>>>>>> 
>>>>>> I'm +1 on this, and can help with the technical details.  As it
>>>>>> stands, all unit conversions happen in a single place in the code, so
>>>>>> we can simply remove the items there and instead just wrap them in the
>>>>>> appropriate "code" units.  We'll need to figure out the specific
>>>>>> semantics of that but it should be okay.  How do we handle the fact
>>>>>> that converting "code_density" to cgs is different for different
>>>>>> arrays, depending on the code?  i.e., if I have two "code_density"
>>>>>> arrays from a code that stores them in comoving code units, how do we
>>>>>> distinguish between those two things during division, say?
>>>>>> 
>>>>>> -Matt
>>>>>> 
>>>>>> On Mon, May 6, 2013 at 6:50 PM, Casey W. Stark <caseywstark at gmail.com>
>>>>>> wrote:
>>>>>>> Hey Nathan.
>>>>>>> 
>>>>>>> Thanks for the reminder about `ensure_cgs`. Definitely helps to see
>>>>>>> the
>>>>>>> pseudocode too. Users could also write out the array.convert_to(...)
>>>>>>> in
>>>>>>> the
>>>>>>> field function for whatever obscure units. I guess this is similar
>>>>>>> to
>>>>>>> the
>>>>>>> fields with the units tacked on to the name, but much cleaner.
>>>>>>> 
>>>>>>> I think making the code to universal field mapping more explicit
>>>>>>> will be
>>>>>>> very helpful. I've been confused about where some derived fields
>>>>>>> come
>>>>>>> from
>>>>>>> many times!
>>>>>>> 
>>>>>>> 
>>>>>>> On Mon, May 6, 2013 at 3:39 PM, Nathan Goldbaum
>>>>>>> <goldbaum at ucolick.org>
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Hey Casey,
>>>>>>>> 
>>>>>>>> That's pretty much what I was thinking, although I wanted to add a
>>>>>>>> little
>>>>>>>> bit of syntactic sugar so the CGS conversion happens automatically:
>>>>>>>> 
>>>>>>>> http://paste.yt-project.org/show/3443/
>>>>>>>> 
>>>>>>>> The ensure_cgs decorator would simply call convert_to_cgs on the
>>>>>>>> returned
>>>>>>>> YTArray.  The nice thing about this is the flow of data is clear,
>>>>>>>> things
>>>>>>>> come in to the field definition code units and leave in CGS.  This
>>>>>>>> also
>>>>>>>> eliminates the need for the unit conversion functions that are
>>>>>>>> scattered
>>>>>>>> throughout the frontends and universal_fields.py.
>>>>>>>> 
>>>>>>>> This has the nice feature that each of the code frontends will need
>>>>>>>> to
>>>>>>>> explicitly define mappings to the CGS fields (like Density) needed
>>>>>>>> by
>>>>>>>> universal fields.
>>>>>>>> 
>>>>>>>> -Nathan
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Mon, May 6, 2013 at 3:33 PM, Casey W. Stark
>>>>>>>> <caseywstark at gmail.com>
>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Hey yt.
>>>>>>>>> 
>>>>>>>>> Nathan and I had a cool idea as an extension of the units work,
>>>>>>>>> but I
>>>>>>>>> am
>>>>>>>>> hitting a yt internals knowledge wall writing a YTEP. I would like
>>>>>>>>> some
>>>>>>>>> input on technical details or general feelings before I move ahead
>>>>>>>>> with the
>>>>>>>>> write-up.
>>>>>>>>> 
>>>>>>>>> One of the things mentioned in the units YTEP is handling code
>>>>>>>>> units.
>>>>>>>>> Basically each frontend static output should register the code
>>>>>>>>> units.
>>>>>>>>> In Nyx
>>>>>>>>> for instance, we would do something like registering "code_length"
>>>>>>>>> as
>>>>>>>>> Mpccm,
>>>>>>>>> "code_mass" as Msun, etc. Then any field from the dataset can be
>>>>>>>>> loaded in
>>>>>>>>> code units with something like
>>>>>>>>> dd["density"].in_units("code_density").
>>>>>>>>> 
>>>>>>>>> This got us thinking -- why load native fields in CGS now?
>>>>>>>>> Instead, we
>>>>>>>>> can define native fields exactly as they are on disk and users who
>>>>>>>>> want code
>>>>>>>>> units will never have to convert by hand again. If frontend
>>>>>>>>> developers
>>>>>>>>> define the fields and units on disk, and some mapping to the
>>>>>>>>> universal
>>>>>>>>> fields, we shouldn't have to define or convert units anywhere
>>>>>>>>> else.
>>>>>>>>> 
>>>>>>>>> I made a quick and dirty example of what we're thinking here:
>>>>>>>>> http://paste.yt-project.org/show/3441/
>>>>>>>>> 
>>>>>>>>> So, is something like this a good idea? Is it realistic? Would
>>>>>>>>> this
>>>>>>>>> make
>>>>>>>>> for a simpler field info container? Nathan, is there anything I
>>>>>>>>> missed?
>>>>>>>>> 
>>>>>>>>> - Casey
>>>>>>>>> 
>>>>>>>>> _______________________________________________
>>>>>>>>> yt-dev mailing list
>>>>>>>>> yt-dev at lists.spacepope.org
>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> yt-dev mailing list
>>>>>>>> yt-dev at lists.spacepope.org
>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> yt-dev mailing list
>>>>>>> yt-dev at lists.spacepope.org
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>>> 
>>>>>> _______________________________________________
>>>>>> yt-dev mailing list
>>>>>> yt-dev at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>> 
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> yt-dev mailing list
>>>>> yt-dev at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>> 
>>>> _______________________________________________
>>>> yt-dev mailing list
>>>> yt-dev at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>> 
>>> 
>> 
>> 
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> 
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>

More information about the yt-dev mailing list