[yt-dev] ytdata frontend work

John ZuHone jzuhone at gmail.com
Tue Oct 27 08:28:53 PDT 2015 

Matt:

http://nbviewer.ipython.org/gist/jzuhone/469dad66c640c2fe25e2 <http://nbviewer.ipython.org/gist/jzuhone/469dad66c640c2fe25e2>

> On Oct 27, 2015, at 11:05 AM, Matthew Turk <matthewturk at gmail.com> wrote:
> 
> Hi Britton,
> 
> Yes -- in fact, I am going to try adding it to your PR.  Is there a
> notebook or script that demonstrates how to make the setup fail as-is?
> 
> On Tue, Oct 27, 2015 at 10:04 AM, Britton Smith <brittonsmith at gmail.com> wrote:
>> Hi Matt,
>> 
>> If we could do that, it would be the cleanest and most effective solution.
>> When you have time, can you lay out how this can be done?
>> 
>> Thanks,
>> Britton
>> 
>> On Tue, Oct 27, 2015 at 2:56 PM, Matthew Turk <matthewturk at gmail.com> wrote:
>>> 
>>> Hi Britton,
>>> 
>>> I've thought about this a bit, and I think this can be addressed.  In
>>> the ytdata frontend, we should be able to make it recognize the index
>>> fields; if we can't, we are overdue for having coordinate handlers
>>> check things out.  There should be logic that prevents derived fields
>>> overriding on-disk fields, so if there is an on-disk field that
>>> presents "index" it should work.
>>> 
>>> On Tue, Oct 27, 2015 at 9:35 AM, Britton Smith <brittonsmith at gmail.com>
>>> wrote:
>>>> Hi all,
>>>> 
>>>> I have spent a while looking at the issue involving the derived
>>>> quantities
>>>> and I have come up with a solution for some of them, but not all of
>>>> them.  I
>>>> have pushed a couple changes to the PR that add a cell_volume and
>>>> cell_mass
>>>> field specific to YTDataContainerDatasets that makes use of their "dx",
>>>> "dy", and "dz" fields.  As of now, the following derived quantities
>>>> work:
>>>> - min location
>>>> - max location
>>>> - extrema
>>>> - bulk velocity
>>>> - total mass
>>>> - weighted average quantity
>>>> - weighted variance
>>>> - total quantity
>>>> 
>>>> The ones that don't work are:
>>>> - spin parameter
>>>> - center of mass
>>>> - angular momentum vector
>>>> 
>>>> The final three all don't work for the same reason, which is that they
>>>> require access to the "index" position fields.  Unfortunately, these
>>>> cannot
>>>> be overridden for reasons that have to do with the _determine_fields
>>>> function.  I have made a point during this development process to alter
>>>> as
>>>> little as possible outside of the ytdata frontend.  However, at this
>>>> point,
>>>> there is no more room to maneuver.  The only solutions I can come up
>>>> with
>>>> are ugly and would make it very difficult to smoothly swap in a proper
>>>> AMR
>>>> hierarchy when we get around to creating one and doing this properly.
>>>> 
>>>> The last thing I can think of would be to swap in special derived
>>>> quantity
>>>> functions for this frontend only that use position fields appropriate
>>>> for
>>>> the frontend.  The functions themselves would be simple to implement,
>>>> but
>>>> because the derived quantity collection is added onto data container
>>>> objects
>>>> and not dataset objects, I have no idea how one would do the swap.  For
>>>> a
>>>> similar reason, I don't know how to simply not allow these problem cases
>>>> to
>>>> be called, or to fail in a clean way for them.
>>>> 
>>>> If anyone has any ideas, I would love to hear them.
>>>> 
>>>> Britton
>>>> 
>>>> On Mon, Oct 26, 2015 at 10:12 PM, John ZuHone <jzuhone at gmail.com> wrote:
>>>>> 
>>>>> No worries—the main issues now are the tests (which don’t run under
>>>>> Python
>>>>> 2) and Python 3.
>>>>> 
>>>>> Tomorrow I’ll point you at the head that I’ve been making changes on to
>>>>> address these.
>>>>> 
>>>>> On Oct 26, 2015, at 6:09 PM, Britton Smith <brittonsmith at gmail.com>
>>>>> wrote:
>>>>> 
>>>>> Hi John,
>>>>> 
>>>>> At the PR hangout, we discussed the possibility of eventually creating
>>>>> a
>>>>> method to extract subsets of AMR hierarchies that would include only
>>>>> grid
>>>>> patches that are at least partially contained within the data
>>>>> container.
>>>>> With them, we would also include the mask arrays to be applied to each
>>>>> grid
>>>>> patch such that you would only get the cells within the container.
>>>>> However,
>>>>> this was deemed something that should not hold up this PR, but that
>>>>> could
>>>>> eventually subbed in to fill in any gaps in functionality created by
>>>>> treating the grid data like particles.  We decided that patching the
>>>>> existing machinery with a frontend-specific cell_volume field was the
>>>>> best
>>>>> thing as it would get this on front of users faster, especially since
>>>>> it
>>>>> contains a lot of other things that are fully working.  I may have
>>>>> missed
>>>>> some important parts of the discussion, but my memory faded a lot
>>>>> faster
>>>>> than I thought it would.
>>>>> 
>>>>> I probably won't have much time to work on this until Friday, but I'll
>>>>> try
>>>>> to play around with the derived quantities when I can.
>>>>> 
>>>>> Britton
>>>>> 
>>>>> On Mon, Oct 26, 2015 at 9:45 PM, John ZuHone <jzuhone at gmail.com> wrote:
>>>>>> 
>>>>>> Hi all,
>>>>>> 
>>>>>> Matt asked someone to send a quick note about the current issues
>>>>>> regarding the ytdata frontend PR:
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> https://bitbucket.org/yt_analysis/yt/pull-requests/1788/adding-ytdata-frontend
>>>>>> 
>>>>>> I’m sending it because I’m working on it currently, but I was not able
>>>>>> to
>>>>>> attend today’s PR hangout so anyone else who was more to add or to
>>>>>> correct
>>>>>> should chime in.
>>>>>> 
>>>>>> Currently, there are 3 issues, from most important to least important:
>>>>>> 
>>>>>> 1. Tests don’t work
>>>>>> 2. Doesn’t support Python 3
>>>>>> 3. Derived quantities don’t work
>>>>>> 
>>>>>> I’m currently working on 1 and 2, and there have been some ideas about
>>>>>> 3.
>>>>>> The main issue with the latter is that for non-gridded data containers
>>>>>> (like
>>>>>> sphere, disk, etc.) created from grid datasets, we treat the new
>>>>>> datasets as
>>>>>> “particle data”. For that reason there is an issue with calculating
>>>>>> the
>>>>>> cell_volume properly. Without that, we cannot use the total_mass()
>>>>>> quantity.
>>>>>> Given that we have the cell deltas in the dataset, we could have a
>>>>>> special
>>>>>> derived field for the cell_volume.
>>>>>> 
>>>>>> There are other issues with some of the other derived quantities
>>>>>> (e.g.,
>>>>>> weighted_average_quantity returning the wrong units), but I haven’t
>>>>>> been
>>>>>> able to diagnose them yet.
>>>>>> 
>>>>>> I’m working on some changes that will need to be merged into Britton’s
>>>>>> fork eventually to address at least 1 and 2.
>>>>>> 
>>>>>> Best,
>>>>>> 
>>>>>> John
>>>>>> 
>>>>>> _______________________________________________
>>>>>> yt-dev mailing list
>>>>>> yt-dev at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> yt-dev mailing list
>>>>> yt-dev at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>> 
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> yt-dev mailing list
>>>>> yt-dev at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> yt-dev mailing list
>>>> yt-dev at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>> 
>>> _______________________________________________
>>> yt-dev mailing list
>>> yt-dev at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> 
>> 
>> 
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> 
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20151027/1792f04d/attachment.html>

More information about the yt-dev mailing list