[yt-dev] Native field discussion

j s oishi jsoishi at gmail.com
Tue May 7 14:14:01 PDT 2013


Yeah, I now see this was a stupid concern. Disregard.

sorry.

j


On Tue, May 7, 2013 at 5:10 PM, Matthew Turk <matthewturk at gmail.com> wrote:

> On Tue, May 7, 2013 at 5:06 PM, j s oishi <jsoishi at gmail.com> wrote:
> >
> >
> >
> > On Tue, May 7, 2013 at 1:25 PM, Matthew Turk <matthewturk at gmail.com>
> wrote:
> >>
> >> On Tue, May 7, 2013 at 1:24 PM, Casey W. Stark <caseywstark at gmail.com>
> >> wrote:
> >> > Ok, I just tried this and it doesn't work. Even though the Unit
> objects
> >> > are
> >> > separate instances, the sympy Symbol objects in the expressions are
> >> > treated
> >> > as the same. We will have to think about the design some more.
> >>
> >> Although, on reflection, maybe *this is okay*.  We should perhaps
> >> assume that anyone who is actually dividing or multiplying or
> >> something the native code units knows what they're doing.  If they
> >> want to look at the cgs units, they should be doing so.  I'm not
> >> entirely sure.
> >
> >
> > I'm not so sure about this. What about simulations that are performed in
> > dimensionless units? This is something that yt handles poorly right now,
> and
> > I think this system should fix that. For example, in a dimensionless
> system,
> > the code units are fully functional and probably what you want to work
> in.
> > Or am I missing something here?
>
> What you're describing will be much, much easier to do in the system
> Casey and Nathan have outlined.  What I was describing was comparing
> two code unitful arrays with the same name where that name means
> something different for each of the two arrays.  For instance,
> comparing the same Enzo cosmology run in *code units* at z=10 and z=1.
>  I at first thought that this would be an undefined operation that we
> don't know how to manage, but then I realized that anyone who is doing
> this probably knows what they are doing.
>
> For dimensionless simulations, you will be able to access the native
> fields with less overhead in this new system, simply by querying -- as
> Casey showed above -- the native fields.
>
> This will also simplify (as Casey noted) the field generation and
> field info container logic.
>
> >
> >>
> >>
> >> >
> >> >
> >> > On Tue, May 7, 2013 at 10:17 AM, Casey W. Stark <
> caseywstark at gmail.com>
> >> > wrote:
> >> >>
> >> >> > think of having two different parameter
> >> >> files from the same simulation open at the same time.
> >> >>
> >> >> Ah, got it. You're right that this will be a problem. The two
> datasets
> >> >> will have separate unit registries, so something like "pccm" units
> can
> >> >> be
> >> >> different from each. However, if you try something like
> >> >> ds1_dd["enzo_comoving_density"] / ds2_dd["enzo_comoving_density"],
> >> >> sympy
> >> >> might just cancel out the symbols in the unit expressions and the
> >> >> conversion
> >> >> will not work. I'll try something...
> >> >>
> >> >>
> >> >> On Tue, May 7, 2013 at 10:05 AM, Matthew Turk <matthewturk at gmail.com
> >
> >> >> wrote:
> >> >>>
> >> >>> On Tue, May 7, 2013 at 12:13 PM, Casey W. Stark
> >> >>> <caseywstark at gmail.com>
> >> >>> wrote:
> >> >>> > Hey Matt.
> >> >>> >
> >> >>> > Excellent! It's a considerable change, so we should dread it a
> bit,
> >> >>> > but
> >> >>> > really glad you like the idea too.
> >> >>> >
> >> >>> > About where the conversion should happen, I like the idea to
> return
> >> >>> > any
> >> >>> > universal fields in cgs by default. Operations between "code"
> fields
> >> >>> > ("mycode_density") would return in code units and ops between
> >> >>> > universal
> >> >>> > fields ("density") would return in cgs. That means most users
> won't
> >> >>> > see
> >> >>> > a
> >> >>> > change too. Mixing units is mostly straightforward. I think the
> only
> >> >>> > gotcha
> >> >>> > is for add and sub, where we return in the left object's units.
> >> >>>
> >> >>> That sounds good to me.
> >> >>>
> >> >>> >
> >> >>> >> How do we handle the fact
> >> >>> > that converting "code_density" to cgs is different for different
> >> >>> > arrays, depending on the code?  i.e., if I have two "code_density"
> >> >>> > arrays from a code that stores them in comoving code units, how do
> >> >>> > we
> >> >>> > distinguish between those two things during division, say?
> >> >>> >
> >> >>> > I'm not sure I'm following here, but is this a case where density
> >> >>> > field
> >> >>> > a
> >> >>> > has different units from density field b? If so, I think we have
> to
> >> >>> > define
> >> >>> > separate units for the code densities. Here's a quick Nyx
> example...
> >> >>>
> >> >>> Not exactly, but close -- think of having two different parameter
> >> >>> files from the same simulation open at the same time.  If the sim
> >> >>> stores internally in comoving units, the native->propercgs will vary
> >> >>> between those two, but the registered units might be the same name,
> >> >>> for instance "enzo_density" or something.
> >> >>>
> >> >>> >
> >> >>> > unit_registry.add("nyx_cm_density", "Msun/Mpccm**3")
> >> >>> > unit_registry.add("nyx_prop_density", "g/cm**3")
> >> >>> >
> >> >>> > add_field("nyx_density", units="nyx_cm_density")
> >> >>> > add_field("nyx_hi_density", units="nyx_prop_density")
> >> >>> >
> >> >>> > dd["nyx_density"] / dd["nyx_hi_density"]    (should return
> >> >>> > dimensionless
> >> >>> > YTArray)
> >> >>> >
> >> >>> > - Casey
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> > On Tue, May 7, 2013 at 8:18 AM, Matthew Turk <
> matthewturk at gmail.com>
> >> >>> > wrote:
> >> >>> >>
> >> >>> >> Hi Casey and Nathan,
> >> >>> >>
> >> >>> >> Last night when I read this I felt a bit of dread in my stomach,
> >> >>> >> worrying that this would make things a lot more complex and
> >> >>> >> difficult.
> >> >>> >>  But I have to say, now having thought it through and read your
> >> >>> >> pastes, reflected on the contents of the YTEP ... I think this is
> >> >>> >> beautiful and elegant.  Very nicely thought out.
> >> >>> >>
> >> >>> >> I completely agree that -- particularly in 3.0 -- we no longer
> need
> >> >>> >> to
> >> >>> >> perform any conversions at IO time, and instead we should be
> >> >>> >> pushing
> >> >>> >> this to the places where the data will be presented to individual
> >> >>> >> users.  This will require some thought about where we want this
> >> >>> >> conversion to take place (i.e., at the display layer?) but it
> >> >>> >> should
> >> >>> >> be completely workable to do so.  The ensure_cgs decorator is a
> >> >>> >> great
> >> >>> >> way to manage this.
> >> >>> >>
> >> >>> >> I'm +1 on this, and can help with the technical details.  As it
> >> >>> >> stands, all unit conversions happen in a single place in the
> code,
> >> >>> >> so
> >> >>> >> we can simply remove the items there and instead just wrap them
> in
> >> >>> >> the
> >> >>> >> appropriate "code" units.  We'll need to figure out the specific
> >> >>> >> semantics of that but it should be okay.  How do we handle the
> fact
> >> >>> >> that converting "code_density" to cgs is different for different
> >> >>> >> arrays, depending on the code?  i.e., if I have two
> "code_density"
> >> >>> >> arrays from a code that stores them in comoving code units, how
> do
> >> >>> >> we
> >> >>> >> distinguish between those two things during division, say?
> >> >>> >>
> >> >>> >> -Matt
> >> >>> >>
> >> >>> >> On Mon, May 6, 2013 at 6:50 PM, Casey W. Stark
> >> >>> >> <caseywstark at gmail.com>
> >> >>> >> wrote:
> >> >>> >> > Hey Nathan.
> >> >>> >> >
> >> >>> >> > Thanks for the reminder about `ensure_cgs`. Definitely helps to
> >> >>> >> > see
> >> >>> >> > the
> >> >>> >> > pseudocode too. Users could also write out the
> >> >>> >> > array.convert_to(...)
> >> >>> >> > in
> >> >>> >> > the
> >> >>> >> > field function for whatever obscure units. I guess this is
> >> >>> >> > similar
> >> >>> >> > to
> >> >>> >> > the
> >> >>> >> > fields with the units tacked on to the name, but much cleaner.
> >> >>> >> >
> >> >>> >> > I think making the code to universal field mapping more
> explicit
> >> >>> >> > will be
> >> >>> >> > very helpful. I've been confused about where some derived
> fields
> >> >>> >> > come
> >> >>> >> > from
> >> >>> >> > many times!
> >> >>> >> >
> >> >>> >> >
> >> >>> >> > On Mon, May 6, 2013 at 3:39 PM, Nathan Goldbaum
> >> >>> >> > <goldbaum at ucolick.org>
> >> >>> >> > wrote:
> >> >>> >> >>
> >> >>> >> >> Hey Casey,
> >> >>> >> >>
> >> >>> >> >> That's pretty much what I was thinking, although I wanted to
> add
> >> >>> >> >> a
> >> >>> >> >> little
> >> >>> >> >> bit of syntactic sugar so the CGS conversion happens
> >> >>> >> >> automatically:
> >> >>> >> >>
> >> >>> >> >> http://paste.yt-project.org/show/3443/
> >> >>> >> >>
> >> >>> >> >> The ensure_cgs decorator would simply call convert_to_cgs on
> the
> >> >>> >> >> returned
> >> >>> >> >> YTArray.  The nice thing about this is the flow of data is
> >> >>> >> >> clear,
> >> >>> >> >> things
> >> >>> >> >> come in to the field definition code units and leave in CGS.
> >> >>> >> >> This
> >> >>> >> >> also
> >> >>> >> >> eliminates the need for the unit conversion functions that are
> >> >>> >> >> scattered
> >> >>> >> >> throughout the frontends and universal_fields.py.
> >> >>> >> >>
> >> >>> >> >> This has the nice feature that each of the code frontends will
> >> >>> >> >> need
> >> >>> >> >> to
> >> >>> >> >> explicitly define mappings to the CGS fields (like Density)
> >> >>> >> >> needed
> >> >>> >> >> by
> >> >>> >> >> universal fields.
> >> >>> >> >>
> >> >>> >> >> -Nathan
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >> On Mon, May 6, 2013 at 3:33 PM, Casey W. Stark
> >> >>> >> >> <caseywstark at gmail.com>
> >> >>> >> >> wrote:
> >> >>> >> >>>
> >> >>> >> >>> Hey yt.
> >> >>> >> >>>
> >> >>> >> >>> Nathan and I had a cool idea as an extension of the units
> work,
> >> >>> >> >>> but I
> >> >>> >> >>> am
> >> >>> >> >>> hitting a yt internals knowledge wall writing a YTEP. I would
> >> >>> >> >>> like
> >> >>> >> >>> some
> >> >>> >> >>> input on technical details or general feelings before I move
> >> >>> >> >>> ahead
> >> >>> >> >>> with the
> >> >>> >> >>> write-up.
> >> >>> >> >>>
> >> >>> >> >>> One of the things mentioned in the units YTEP is handling
> code
> >> >>> >> >>> units.
> >> >>> >> >>> Basically each frontend static output should register the
> code
> >> >>> >> >>> units.
> >> >>> >> >>> In Nyx
> >> >>> >> >>> for instance, we would do something like registering
> >> >>> >> >>> "code_length"
> >> >>> >> >>> as
> >> >>> >> >>> Mpccm,
> >> >>> >> >>> "code_mass" as Msun, etc. Then any field from the dataset can
> >> >>> >> >>> be
> >> >>> >> >>> loaded in
> >> >>> >> >>> code units with something like
> >> >>> >> >>> dd["density"].in_units("code_density").
> >> >>> >> >>>
> >> >>> >> >>> This got us thinking -- why load native fields in CGS now?
> >> >>> >> >>> Instead, we
> >> >>> >> >>> can define native fields exactly as they are on disk and
> users
> >> >>> >> >>> who
> >> >>> >> >>> want code
> >> >>> >> >>> units will never have to convert by hand again. If frontend
> >> >>> >> >>> developers
> >> >>> >> >>> define the fields and units on disk, and some mapping to the
> >> >>> >> >>> universal
> >> >>> >> >>> fields, we shouldn't have to define or convert units anywhere
> >> >>> >> >>> else.
> >> >>> >> >>>
> >> >>> >> >>> I made a quick and dirty example of what we're thinking here:
> >> >>> >> >>> http://paste.yt-project.org/show/3441/
> >> >>> >> >>>
> >> >>> >> >>> So, is something like this a good idea? Is it realistic?
> Would
> >> >>> >> >>> this
> >> >>> >> >>> make
> >> >>> >> >>> for a simpler field info container? Nathan, is there
> anything I
> >> >>> >> >>> missed?
> >> >>> >> >>>
> >> >>> >> >>> - Casey
> >> >>> >> >>>
> >> >>> >> >>> _______________________________________________
> >> >>> >> >>> yt-dev mailing list
> >> >>> >> >>> yt-dev at lists.spacepope.org
> >> >>> >> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>> >> >>>
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >> _______________________________________________
> >> >>> >> >> yt-dev mailing list
> >> >>> >> >> yt-dev at lists.spacepope.org
> >> >>> >> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>> >> >>
> >> >>> >> >
> >> >>> >> >
> >> >>> >> > _______________________________________________
> >> >>> >> > yt-dev mailing list
> >> >>> >> > yt-dev at lists.spacepope.org
> >> >>> >> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>> >> >
> >> >>> >> _______________________________________________
> >> >>> >> yt-dev mailing list
> >> >>> >> yt-dev at lists.spacepope.org
> >> >>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> > _______________________________________________
> >> >>> > yt-dev mailing list
> >> >>> > yt-dev at lists.spacepope.org
> >> >>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>> >
> >> >>> _______________________________________________
> >> >>> yt-dev mailing list
> >> >>> yt-dev at lists.spacepope.org
> >> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>
> >> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > yt-dev mailing list
> >> > yt-dev at lists.spacepope.org
> >> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >
> >> _______________________________________________
> >> yt-dev mailing list
> >> yt-dev at lists.spacepope.org
> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >
> >
> >
> > _______________________________________________
> > yt-dev mailing list
> > yt-dev at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20130507/23ee4ebd/attachment.htm>


More information about the yt-dev mailing list