[yt-dev] ytdata frontend work

[Matthew Turk]

Hi Britton,

Yes -- in fact, I am going to try adding it to your PR.  Is there a
notebook or script that demonstrates how to make the setup fail as-is?

On Tue, Oct 27, 2015 at 10:04 AM, Britton Smith <brittonsmith at gmail.com> wrote:
> Hi Matt,
>
> If we could do that, it would be the cleanest and most effective solution.
> When you have time, can you lay out how this can be done?
>
> Thanks,
> Britton
>
> On Tue, Oct 27, 2015 at 2:56 PM, Matthew Turk <matthewturk at gmail.com> wrote:
>>
>> Hi Britton,
>>
>> I've thought about this a bit, and I think this can be addressed.  In
>> the ytdata frontend, we should be able to make it recognize the index
>> fields; if we can't, we are overdue for having coordinate handlers
>> check things out.  There should be logic that prevents derived fields
>> overriding on-disk fields, so if there is an on-disk field that
>> presents "index" it should work.
>>
>> On Tue, Oct 27, 2015 at 9:35 AM, Britton Smith <brittonsmith at gmail.com>
>> wrote:
>> > Hi all,
>> >
>> > I have spent a while looking at the issue involving the derived
>> > quantities
>> > and I have come up with a solution for some of them, but not all of
>> > them.  I
>> > have pushed a couple changes to the PR that add a cell_volume and
>> > cell_mass
>> > field specific to YTDataContainerDatasets that makes use of their "dx",
>> > "dy", and "dz" fields.  As of now, the following derived quantities
>> > work:
>> > - min location
>> > - max location
>> > - extrema
>> > - bulk velocity
>> > - total mass
>> > - weighted average quantity
>> > - weighted variance
>> > - total quantity
>> >
>> > The ones that don't work are:
>> > - spin parameter
>> > - center of mass
>> > - angular momentum vector
>> >
>> > The final three all don't work for the same reason, which is that they
>> > require access to the "index" position fields.  Unfortunately, these
>> > cannot
>> > be overridden for reasons that have to do with the _determine_fields
>> > function.  I have made a point during this development process to alter
>> > as
>> > little as possible outside of the ytdata frontend.  However, at this
>> > point,
>> > there is no more room to maneuver.  The only solutions I can come up
>> > with
>> > are ugly and would make it very difficult to smoothly swap in a proper
>> > AMR
>> > hierarchy when we get around to creating one and doing this properly.
>> >
>> > The last thing I can think of would be to swap in special derived
>> > quantity
>> > functions for this frontend only that use position fields appropriate
>> > for
>> > the frontend.  The functions themselves would be simple to implement,
>> > but
>> > because the derived quantity collection is added onto data container
>> > objects
>> > and not dataset objects, I have no idea how one would do the swap.  For
>> > a
>> > similar reason, I don't know how to simply not allow these problem cases
>> > to
>> > be called, or to fail in a clean way for them.
>> >
>> > If anyone has any ideas, I would love to hear them.
>> >
>> > Britton
>> >
>> > On Mon, Oct 26, 2015 at 10:12 PM, John ZuHone <jzuhone at gmail.com> wrote:
>> >>
>> >> No worries—the main issues now are the tests (which don’t run under
>> >> Python
>> >> 2) and Python 3.
>> >>
>> >> Tomorrow I’ll point you at the head that I’ve been making changes on to
>> >> address these.
>> >>
>> >> On Oct 26, 2015, at 6:09 PM, Britton Smith <brittonsmith at gmail.com>
>> >> wrote:
>> >>
>> >> Hi John,
>> >>
>> >> At the PR hangout, we discussed the possibility of eventually creating
>> >> a
>> >> method to extract subsets of AMR hierarchies that would include only
>> >> grid
>> >> patches that are at least partially contained within the data
>> >> container.
>> >> With them, we would also include the mask arrays to be applied to each
>> >> grid
>> >> patch such that you would only get the cells within the container.
>> >> However,
>> >> this was deemed something that should not hold up this PR, but that
>> >> could
>> >> eventually subbed in to fill in any gaps in functionality created by
>> >> treating the grid data like particles.  We decided that patching the
>> >> existing machinery with a frontend-specific cell_volume field was the
>> >> best
>> >> thing as it would get this on front of users faster, especially since
>> >> it
>> >> contains a lot of other things that are fully working.  I may have
>> >> missed
>> >> some important parts of the discussion, but my memory faded a lot
>> >> faster
>> >> than I thought it would.
>> >>
>> >> I probably won't have much time to work on this until Friday, but I'll
>> >> try
>> >> to play around with the derived quantities when I can.
>> >>
>> >> Britton
>> >>
>> >> On Mon, Oct 26, 2015 at 9:45 PM, John ZuHone <jzuhone at gmail.com> wrote:
>> >>>
>> >>> Hi all,
>> >>>
>> >>> Matt asked someone to send a quick note about the current issues
>> >>> regarding the ytdata frontend PR:
>> >>>
>> >>>
>> >>>
>> >>> https://bitbucket.org/yt_analysis/yt/pull-requests/1788/adding-ytdata-frontend
>> >>>
>> >>> I’m sending it because I’m working on it currently, but I was not able
>> >>> to
>> >>> attend today’s PR hangout so anyone else who was more to add or to
>> >>> correct
>> >>> should chime in.
>> >>>
>> >>> Currently, there are 3 issues, from most important to least important:
>> >>>
>> >>> 1. Tests don’t work
>> >>> 2. Doesn’t support Python 3
>> >>> 3. Derived quantities don’t work
>> >>>
>> >>> I’m currently working on 1 and 2, and there have been some ideas about
>> >>> 3.
>> >>> The main issue with the latter is that for non-gridded data containers
>> >>> (like
>> >>> sphere, disk, etc.) created from grid datasets, we treat the new
>> >>> datasets as
>> >>> “particle data”. For that reason there is an issue with calculating
>> >>> the
>> >>> cell_volume properly. Without that, we cannot use the total_mass()
>> >>> quantity.
>> >>> Given that we have the cell deltas in the dataset, we could have a
>> >>> special
>> >>> derived field for the cell_volume.
>> >>>
>> >>> There are other issues with some of the other derived quantities
>> >>> (e.g.,
>> >>> weighted_average_quantity returning the wrong units), but I haven’t
>> >>> been
>> >>> able to diagnose them yet.
>> >>>
>> >>> I’m working on some changes that will need to be merged into Britton’s
>> >>> fork eventually to address at least 1 and 2.
>> >>>
>> >>> Best,
>> >>>
>> >>> John
>> >>>
>> >>> _______________________________________________
>> >>> yt-dev mailing list
>> >>> yt-dev at lists.spacepope.org
>> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>>
>> >>
>> >> _______________________________________________
>> >> yt-dev mailing list
>> >> yt-dev at lists.spacepope.org
>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> yt-dev mailing list
>> >> yt-dev at lists.spacepope.org
>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>
>> >
>> >
>> > _______________________________________________
>> > yt-dev mailing list
>> > yt-dev at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>