[yt-dev] yt-3.0 derived fields

Casey W. Stark caseywstark at gmail.com
Thu Mar 14 10:12:17 PDT 2013 

>
> Note, this will almost certainly fail since the number of star particles
> won't equal the
> number of dark matter particles.


Whoops! Good catch. I hope the example is useful to get a sense for this
otherwise. You could do something like in_units("Msun") instead.

- Casey


On Thu, Mar 14, 2013 at 9:59 AM, Douglas Harvey Rudd <drudd at uchicago.edu>wrote:

>
>  star_particle_masses = dd[("stars", "mass")]
> dm_particle_masses = dd[("dark_matter", "mass")]
> assert (star_particle_masses < dm_particle_masses).all()
>
>
>  Note, this will almost certainly fail since the number of star particles
> won't equal the
> number of dark matter particles.
>
>
>  Being consistent here, I also say we rename "CellMass" to "mass". This
> means you would need dd[("gas", "mass")] and dd["mass"] would fail because
> it is ambiguous. I would prefer really simple field names, but more
> explicit field "type" selectors. Maybe that's just me.
>
>
>  I'm +1 on this as well.  I'd rather the code not guess (or be very clear
> when it is allowing an unspecified type), however this will completely
> break backwards compatibility in many cases.  How much do people care about
> that for 3.0?
>
>  Doug
>
>
> On Thu, Mar 14, 2013 at 6:49 AM, Matthew Turk <matthewturk at gmail.com>wrote:
>
>> Hi Doug,
>>
>> On Thu, Mar 14, 2013 at 9:31 AM, Douglas Harvey Rudd <drudd at uchicago.edu>
>> wrote:
>> > Great!
>> >
>> > If you'd like some sample code, clone my branch drudd/yt-3.0-artio and
>> use
>> > the artio reader on particles.
>>
>>  Perfect, I definitely will.
>>
>> >
>> > In that fork I've also experimented with setting up derived fields with
>> an
>> > explicit type, say ParticleMassMsun for each of all, stars, and nobody,
>> but
>> > this seems to break naked requests for data["ParticleMassMsun"], which
>> > translates the type into 'unknown'.
>>
>>  Grr.  Okay, I think this is one of my issues.  The problem I was
>> running into was keeping a fast but definite cache of the previously
>> requested particle type.  I'll dig in on this.
>>
>> >
>> > The star formation spectrum code currently works with untyped fields
>> > dd["creation_time"]. I'd in principle like to change those to
>> > ("star","creation_time") to be explicit and future compatible (avoid
>> > including black holes that also have creation times associated), but
>> that
>> > will currently break frontends that don't have types or know how to
>> > subselect by type.
>>
>>  I agree 100%.  This is an opportunity to also get rid of all of the
>> "creation_time < 0" lines sprinkled throughout the code.  By sticking
>> all of the particle selection code into the frontends, this is much
>> more tractable.
>>
>> >
>> > Another option that would work for artio is have naked requests resolve
>> to
>> > the correct particle type, if only one particle type supports it, or
>> "all"
>> > otherwise.
>>
>>  I think this is good for the most part, but I am worried it may
>> interfere with calls by particle type specific fields that are neutral
>> to the particle type.  For instance, imagine a case where you have a
>> derived field that calls on particle velocity.  If that top level
>> field is called only on stars, then we want to make sure that the call
>> to velocity resolves only to stars.  I think this is tractable, I just
>> want to make sure that we are careful to ensure that still works.
>>
>> FWIW, there is a default_fluid_type attribute.  Perhaps identifying
>> where it is used and then mimicking it for particles would work.
>>
>> -Matt
>>
>> >
>> >
>> > Douglas Rudd
>> > Scientific Computing Consultant
>> > Research Computing Center and
>> > Kavli Center for Cosmological Physics
>> > drudd at uchicago.edu
>> >
>> > On Mar 14, 2013, at 7:08 AM, "Matthew Turk" <matthewturk at gmail.com>
>> wrote:
>> >
>> > Hi Doug,
>> >
>> > On Wed, Mar 13, 2013 at 7:56 PM, Douglas Harvey Rudd <
>> drudd at uchicago.edu>
>> > wrote:
>> >
>> > Okay, I had noticed this thread but hadn't fully digested that I had
>> hit the
>> > very same issue.
>> >
>> >
>> > I'm +1 on Chris' change, for the exact same reason (ParticleMassMsun for
>> > star particle types).
>> >
>> >
>> > Awesome.
>> >
>> >
>> > Matt: I didn't quite follow your message.  I would assume most derived
>> > fields should be neutral
>> >
>> > to the type of particle, since they're typically unit conversions.
>>  Having
>> > two particle types with the
>> >
>> > same field name that are not compatible should be forbidden, otherwise
>> > ("all", ) would be nonsensical.
>> >
>> >
>> > Good point.
>> >
>> >
>> > Why is discriminating particles and fluids desirable here?  By the
>> particle_
>> > naming convention
>> >
>> > for particle fields there shouldn't be any overlap.  I feel that I must
>> have
>> > missed something.
>> >
>> >
>> > There has been some talk of moving away from that.  (It is an
>> > Enzo-ism.)  Although, looks like in Casey's YTEP for field names, we
>> > retain them:
>> >
>> > https://ytep.readthedocs.org/en/latest/YTEPs/YTEP-0003.html
>> >
>> > The only issue I was thinking of was that there may be a time when we
>> > want to do "mass" that applies to both.
>> >
>> > That being said, as long as we keep "all" as the main name, there's no
>> > issue.  So let's mark my concern as unwarranted.
>> >
>> >
>> > In a similar vein, I've changed our GeometryHandler._detect_fields
>> function
>> > to generate all valid
>> >
>> > (particle_type, field) pairs, so that ("nbody","creation_time") is not a
>> > valid field, for example, while
>> >
>> > ("all","particle_mass"), ("nbody","particle_mass"), and
>> > ("stars","particle_mass") are all valid.  This
>> >
>> > has broken the known field support, however, since we currently only add
>> > naked field types,
>> >
>> > without particle types.  It would be a pain to need to add all valid
>> pairs
>> > in fields.py as well.
>> >
>> >
>> > I agree.  I think your solution is right, they should be detected
>> > correctly in that routine.  I'll work on getting the field system
>> > Chris proposed working, which should be a first step toward fixing the
>> > valid pairs issue.
>> >
>> > -Matt
>> >
>> >
>> > Douglas Rudd
>> >
>> > Scientific Computing Consultant
>> >
>> > Research Computing Center
>> >
>> > drudd at uchicago.edu
>> >
>> >
>> >
>> >
>> > On Mar 10, 2013, at 7:21 PM, Matthew Turk <matthewturk at gmail.com>
>> wrote:
>> >
>> >
>> > I'd rather that not work, to tell the truth.  I think Chris's
>> >
>> > suggestion, where you define a derived field that is neutral to the
>> >
>> > *type* of particle, is powerful, but I don't see an advantage to it
>> >
>> > during the call to getting the data, particularly since that loses a
>> >
>> > bit of ability to discriminate between particles/fluids.
>> >
>> >
>> > On Sun, Mar 10, 2013 at 8:14 PM, Nathan Goldbaum <nathan12343 at gmail.com
>> >
>> > wrote:
>> >
>> > Just to clarify, dd[…, "ParticleMassMsun"] would work as well, right?
>> >
>> >
>> > On Mar 10, 2013, at 5:05 PM, Christopher Moody <chrisemoody at gmail.com>
>> >
>> > wrote:
>> >
>> >
>> > Hi Sam,
>> >
>> > Yeah; I think it should change only the derived field. You want the
>> derived
>> >
>> > field to pass along whatever field type you asked for, or use the
>> implicit
>> >
>> > one if you didn't ask for the type explicitly. If you tried to implement
>> >
>> > this in the original call, you'd do something like
>> >
>> > dd[(Ellipsis,"ParticleMassMsun")].   I can't see use cases where this
>> would
>> >
>> > be better than dd[("all","ParticleMassMsun")] -- so yeah, no changes to
>> the
>> >
>> > original call.
>> >
>> >
>> > chris
>> >
>> >
>> >
>> > On Sun, Mar 10, 2013 at 4:30 PM, Sam Skillman <samskillman at gmail.com>
>> wrote:
>> >
>> >
>> > Hi Chris,
>> >
>> >
>> > Just to be clear, this change would occur in the derived field
>> definition,
>> >
>> > not the call to get the star ParticleMassMsun itself, correct?
>> >
>> >
>> > If so, I think something like this, or with a yt-specific term, would be
>> >
>> > great.
>> >
>> >
>> > Thanks,
>> >
>> > Sam
>> >
>> >
>> >
>> > On Sun, Mar 10, 2013 at 5:20 PM, Matthew Turk <matthewturk at gmail.com>
>> >
>> > wrote:
>> >
>> >
>> > Hi Chris,
>> >
>> >
>> > On Sat, Mar 9, 2013 at 4:00 PM, Christopher Moody <
>> chrisemoody at gmail.com>
>> >
>> > wrote:
>> >
>> > Hi everyone,
>> >
>> > Fields in yt-3.0 are now defined like (field_type,field_name) but for
>> >
>> > backwards-compatibility still maintains a field_name ->
>> >
>> > ("all",field_name)
>> >
>> > mapping. Consider a few use cases:
>> >
>> >
>> > keys = ["ParticleMassMsun",
>> >
>> >       ("all","ParticleMassMsun"),
>> >
>> >       ("stars","particle_mass"),
>> >
>> >       ("stars","ParticleMassMsun") ]#broken
>> >
>> >
>> > I think we all have natural expectations for what should happen, but at
>> >
>> > the
>> >
>> > moment the last key in the series is broken. particle_mass is a native
>> >
>> > field, ParticleMassMsun is a derived field:
>> >
>> >
>> > def _ParticleMassMsun(field, data):
>> >
>> >   return data["particle_mass"]/mass_sun_cgs
>> >
>> > add_field("ParticleMassMsun", function=_ParticleMassMsun,
>> >
>> > particle_type=True)
>> >
>> >
>> > Note that in this case data["particle_mass"] maps to
>> >
>> > data[("all","particle_mass")] even when I originally asked for
>> >
>> > dd[("stars","ParticleMassMsun")], which is why things break.
>> >
>> >
>> > So I propose a syntax where in the derived field definition we can do
>> >
>> > something like data[(Ellipsis,"ParticleMassMsun")] or alternatively
>> >
>> > data[(field.field_type,"ParticleMassMsun")]. This ensures that derived
>> >
>> > fields will work intuitively and be as specific to a field type as they
>> >
>> > want
>> >
>> > to be or just pass through whatever field type they may want.
>> >
>> >
>> >
>> > I like this, but maybe we should use a yt-specific object instead of
>> >
>> > Ellipsis.  This would be very nice and is something I have thought /
>> >
>> > worried about and been unable to come up with an elegant solution for.
>> >
>> > Seems like you found one!  :)
>> >
>> >
>> > This week I need to address this for my own work on a simulation with
>> >
>> > multiple particle types; I'll take a pass at implementing something
>> >
>> > like this.  We're also a bit overdue for a YTEP that details access to
>> >
>> > multiple fluids and multiples particles.  Once some things get off my
>> >
>> > plate on Tuesday I will make that a priority.  Then we can use that as
>> >
>> > a place to sort of hash this out and how it'll work.
>> >
>> >
>> > -Matt
>> >
>> >
>> > Thanks!
>> >
>> > chris
>> >
>> >
>> > _______________________________________________
>> >
>> > yt-dev mailing list
>> >
>> > yt-dev at lists.spacepope.org
>> >
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> >
>> > _______________________________________________
>> >
>> > yt-dev mailing list
>> >
>> > yt-dev at lists.spacepope.org
>> >
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> >
>> >
>> >
>> > _______________________________________________
>> >
>> > yt-dev mailing list
>> >
>> > yt-dev at lists.spacepope.org
>> >
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> >
>> >
>> > _______________________________________________
>> >
>> > yt-dev mailing list
>> >
>> > yt-dev at lists.spacepope.org
>> >
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> >
>> >
>> >
>> > _______________________________________________
>> >
>> > yt-dev mailing list
>> >
>> > yt-dev at lists.spacepope.org
>> >
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> >
>> > _______________________________________________
>> >
>> > yt-dev mailing list
>> >
>> > yt-dev at lists.spacepope.org
>> >
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> >
>> > _______________________________________________
>> >
>> > yt-dev mailing list
>> >
>> > yt-dev at lists.spacepope.org
>> >
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> > _______________________________________________
>> > yt-dev mailing list
>> > yt-dev at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> >
>> > _______________________________________________
>> > yt-dev mailing list
>> > yt-dev at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>
>  _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20130314/8dd31dd7/attachment.htm>

More information about the yt-dev mailing list