[yt-dev] ytdata frontend work

Britton Smith brittonsmith at gmail.com
Tue Oct 27 08:32:37 PDT 2015 

I was about to send this one:
http://paste.yt-project.org/show/6000/

Incidentally, it may be better to do this from changeset, 754e08c17816,
which precedes my adding of a ytdata-specific cell_volume function.  In
that case, even this is suitable:
http://paste.yt-project.org/show/6001/

On Tue, Oct 27, 2015 at 3:05 PM, Matthew Turk <matthewturk at gmail.com> wrote:

> Hi Britton,
>
> Yes -- in fact, I am going to try adding it to your PR.  Is there a
> notebook or script that demonstrates how to make the setup fail as-is?
>
> On Tue, Oct 27, 2015 at 10:04 AM, Britton Smith <brittonsmith at gmail.com>
> wrote:
> > Hi Matt,
> >
> > If we could do that, it would be the cleanest and most effective
> solution.
> > When you have time, can you lay out how this can be done?
> >
> > Thanks,
> > Britton
> >
> > On Tue, Oct 27, 2015 at 2:56 PM, Matthew Turk <matthewturk at gmail.com>
> wrote:
> >>
> >> Hi Britton,
> >>
> >> I've thought about this a bit, and I think this can be addressed.  In
> >> the ytdata frontend, we should be able to make it recognize the index
> >> fields; if we can't, we are overdue for having coordinate handlers
> >> check things out.  There should be logic that prevents derived fields
> >> overriding on-disk fields, so if there is an on-disk field that
> >> presents "index" it should work.
> >>
> >> On Tue, Oct 27, 2015 at 9:35 AM, Britton Smith <brittonsmith at gmail.com>
> >> wrote:
> >> > Hi all,
> >> >
> >> > I have spent a while looking at the issue involving the derived
> >> > quantities
> >> > and I have come up with a solution for some of them, but not all of
> >> > them.  I
> >> > have pushed a couple changes to the PR that add a cell_volume and
> >> > cell_mass
> >> > field specific to YTDataContainerDatasets that makes use of their
> "dx",
> >> > "dy", and "dz" fields.  As of now, the following derived quantities
> >> > work:
> >> > - min location
> >> > - max location
> >> > - extrema
> >> > - bulk velocity
> >> > - total mass
> >> > - weighted average quantity
> >> > - weighted variance
> >> > - total quantity
> >> >
> >> > The ones that don't work are:
> >> > - spin parameter
> >> > - center of mass
> >> > - angular momentum vector
> >> >
> >> > The final three all don't work for the same reason, which is that they
> >> > require access to the "index" position fields.  Unfortunately, these
> >> > cannot
> >> > be overridden for reasons that have to do with the _determine_fields
> >> > function.  I have made a point during this development process to
> alter
> >> > as
> >> > little as possible outside of the ytdata frontend.  However, at this
> >> > point,
> >> > there is no more room to maneuver.  The only solutions I can come up
> >> > with
> >> > are ugly and would make it very difficult to smoothly swap in a proper
> >> > AMR
> >> > hierarchy when we get around to creating one and doing this properly.
> >> >
> >> > The last thing I can think of would be to swap in special derived
> >> > quantity
> >> > functions for this frontend only that use position fields appropriate
> >> > for
> >> > the frontend.  The functions themselves would be simple to implement,
> >> > but
> >> > because the derived quantity collection is added onto data container
> >> > objects
> >> > and not dataset objects, I have no idea how one would do the swap.
> For
> >> > a
> >> > similar reason, I don't know how to simply not allow these problem
> cases
> >> > to
> >> > be called, or to fail in a clean way for them.
> >> >
> >> > If anyone has any ideas, I would love to hear them.
> >> >
> >> > Britton
> >> >
> >> > On Mon, Oct 26, 2015 at 10:12 PM, John ZuHone <jzuhone at gmail.com>
> wrote:
> >> >>
> >> >> No worries—the main issues now are the tests (which don’t run under
> >> >> Python
> >> >> 2) and Python 3.
> >> >>
> >> >> Tomorrow I’ll point you at the head that I’ve been making changes on
> to
> >> >> address these.
> >> >>
> >> >> On Oct 26, 2015, at 6:09 PM, Britton Smith <brittonsmith at gmail.com>
> >> >> wrote:
> >> >>
> >> >> Hi John,
> >> >>
> >> >> At the PR hangout, we discussed the possibility of eventually
> creating
> >> >> a
> >> >> method to extract subsets of AMR hierarchies that would include only
> >> >> grid
> >> >> patches that are at least partially contained within the data
> >> >> container.
> >> >> With them, we would also include the mask arrays to be applied to
> each
> >> >> grid
> >> >> patch such that you would only get the cells within the container.
> >> >> However,
> >> >> this was deemed something that should not hold up this PR, but that
> >> >> could
> >> >> eventually subbed in to fill in any gaps in functionality created by
> >> >> treating the grid data like particles.  We decided that patching the
> >> >> existing machinery with a frontend-specific cell_volume field was the
> >> >> best
> >> >> thing as it would get this on front of users faster, especially since
> >> >> it
> >> >> contains a lot of other things that are fully working.  I may have
> >> >> missed
> >> >> some important parts of the discussion, but my memory faded a lot
> >> >> faster
> >> >> than I thought it would.
> >> >>
> >> >> I probably won't have much time to work on this until Friday, but
> I'll
> >> >> try
> >> >> to play around with the derived quantities when I can.
> >> >>
> >> >> Britton
> >> >>
> >> >> On Mon, Oct 26, 2015 at 9:45 PM, John ZuHone <jzuhone at gmail.com>
> wrote:
> >> >>>
> >> >>> Hi all,
> >> >>>
> >> >>> Matt asked someone to send a quick note about the current issues
> >> >>> regarding the ytdata frontend PR:
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> https://bitbucket.org/yt_analysis/yt/pull-requests/1788/adding-ytdata-frontend
> >> >>>
> >> >>> I’m sending it because I’m working on it currently, but I was not
> able
> >> >>> to
> >> >>> attend today’s PR hangout so anyone else who was more to add or to
> >> >>> correct
> >> >>> should chime in.
> >> >>>
> >> >>> Currently, there are 3 issues, from most important to least
> important:
> >> >>>
> >> >>> 1. Tests don’t work
> >> >>> 2. Doesn’t support Python 3
> >> >>> 3. Derived quantities don’t work
> >> >>>
> >> >>> I’m currently working on 1 and 2, and there have been some ideas
> about
> >> >>> 3.
> >> >>> The main issue with the latter is that for non-gridded data
> containers
> >> >>> (like
> >> >>> sphere, disk, etc.) created from grid datasets, we treat the new
> >> >>> datasets as
> >> >>> “particle data”. For that reason there is an issue with calculating
> >> >>> the
> >> >>> cell_volume properly. Without that, we cannot use the total_mass()
> >> >>> quantity.
> >> >>> Given that we have the cell deltas in the dataset, we could have a
> >> >>> special
> >> >>> derived field for the cell_volume.
> >> >>>
> >> >>> There are other issues with some of the other derived quantities
> >> >>> (e.g.,
> >> >>> weighted_average_quantity returning the wrong units), but I haven’t
> >> >>> been
> >> >>> able to diagnose them yet.
> >> >>>
> >> >>> I’m working on some changes that will need to be merged into
> Britton’s
> >> >>> fork eventually to address at least 1 and 2.
> >> >>>
> >> >>> Best,
> >> >>>
> >> >>> John
> >> >>>
> >> >>> _______________________________________________
> >> >>> yt-dev mailing list
> >> >>> yt-dev at lists.spacepope.org
> >> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>>
> >> >>
> >> >> _______________________________________________
> >> >> yt-dev mailing list
> >> >> yt-dev at lists.spacepope.org
> >> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>
> >> >>
> >> >>
> >> >> _______________________________________________
> >> >> yt-dev mailing list
> >> >> yt-dev at lists.spacepope.org
> >> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > yt-dev mailing list
> >> > yt-dev at lists.spacepope.org
> >> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >
> >> _______________________________________________
> >> yt-dev mailing list
> >> yt-dev at lists.spacepope.org
> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >
> >
> >
> > _______________________________________________
> > yt-dev mailing list
> > yt-dev at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20151027/48358a2d/attachment.html>

More information about the yt-dev mailing list