[yt-dev] RFC: Staggered deprecation through 3.1 and 3.2

Cameron Hummels chummels at gmail.com
Wed Jun 25 09:01:02 PDT 2014


Britton, there are also some halo recipes in the cookbook, which I'm not
sure where we want to replace, where we want to cut, etc.  Right now they
aren't working, but I don't know what to do with them:

https://trello.com/c/NJMfoXdl/22-be-able-to-run-the-cookbook

halo_merger_tree.py
halo_plotting.py
halo_profiler.py

This doesn't have to fall at your feet, but if you're looking at the halo
docs, this might be a thing to glance at too.


On Wed, Jun 25, 2014 at 8:54 AM, Britton Smith <brittonsmith at gmail.com>
wrote:

> Nathan, yeah, I can do that.
>
>
> On Wed, Jun 25, 2014 at 4:42 PM, Nathan Goldbaum <nathan12343 at gmail.com>
> wrote:
>
>> There are still some vestiges of the old halo machinery in the docs.
>>  Britton, do you think you'd be up for looking over the halo analysis docs
>> and updating it where necessary?  I'm specifically referring to some of the
>> errors and warnings in the docs build, (see
>> http://tests.yt-project.org/job/yt-docs-3.0/lastSuccessfulBuild/consoleFull,
>> search for "halo") but it probably also deserves a full proofread.
>>
>> I haven't felt comfortable touching the halo finding docs since I'm not
>> familiar with any of the halo analysis code.
>>
>> On Wednesday, June 25, 2014, Britton Smith <brittonsmith at gmail.com>
>> wrote:
>>
>>> Getting all the cookbook recipes working is absolutely a blocker for
>>> 3.0.  I think we're all in agreement on that.  Someone should correct me if
>>> I'm wrong, but all of the existing recipes can and will be made to work by
>>> porting existing functionality to yt-3.0.  This mostly involves adopting
>>> the various newisms, units, field names, etc.  We will not release yt-3.0
>>> with broken functionality.  This is doable on a relatively short timescale
>>> I think.  However, the new VR machinery will not be ready for a bit longer,
>>> so I think it's ok for that to come in 3.1.
>>>
>>> As far as halos are concerned, here's the status there:
>>> - halo_analysis completely replaces the HaloProfiler.  The halo_analysis
>>> functionality is fully documented (Thanks Hilary!) and the HaloProfiler has
>>> been removed from the source and the docs.
>>> - all halo finders have been ported and are now compliant with
>>> halo_analysis (Thanks again to Hilary).
>>> - HaloCatalogTimeSeries is a ways off and will need to be pushed to 3.1
>>> or 3.2.  This is functionality that never existed anywhere, so that's fine.
>>> - the other cards on the halo_analysis board are minor enhancements, not
>>> blockers
>>> - the halo mass function is almost ready and has an open PR that just
>>> needs a few things cleaned up and it's ready.
>>> - the big thing we are missing is a merger tree as neither of the old
>>> ones were ported.  The idea is for them to work with the halo_analysis and
>>> HaloCatalogTimeSeries, which will probably require work akin to a redesign.
>>>  For this reason, this will probably need to wait until 3.1.
>>>
>>> Britton
>>>
>>>
>>> On Wed, Jun 25, 2014 at 2:35 AM, Cameron Hummels <chummels at gmail.com>
>>> wrote:
>>>
>>>> I will continue to work on these, but you're right, we need more people
>>>> working on these little issues or this will continue to be a big blocker
>>>> and not get done.  I can assign specific bugs to the specialists on those
>>>> issues, if it is helpful (see below).
>>>>
>>>> For all issues, see this page:
>>>> https://trello.com/c/NJMfoXdl/22-be-able-to-run-the-cookbook .
>>>>  Reading the description can give you some clues as to the source of the
>>>> error, but the best way to diagnose it is simply to go into the doc
>>>> cookbook directory and try to run the python script:
>>>>
>>>> e.g.
>>>> $ cd $YT_DEST/doc/source/cookbook
>>>> $ python script.py
>>>>
>>>>
>>>>
>>>> * VR - KDTree issues (Sam Skillman? Matt Turk?)
>>>> ----amrkdtree_downsampling.py
>>>> ----camera_movement.py -- something going on with statusbar in brick
>>>> counting!
>>>> ----opaque_rendering.py -- same as camera_movement.py
>>>> ----render_with_box_and_grids.py
>>>>
>>>> * Light Rays and Light Cones (Britton Smith, John Zuhone, Devin Silvia)
>>>> ----make_light_ray.py
>>>> ----unique_light_cone_projections.py
>>>> ----light_cone_projection.py
>>>> ----light_cone_with_halo_mask.py
>>>>
>>>> * Profiles (Matt Turk, Nathan Goldbaum, other?)
>>>> ----global_phase_plots.py
>>>> ----profile_with_variance.py
>>>> ----rad_velocity.py
>>>> ----radial_profile_styles.py?
>>>> ----simple_profile.py
>>>> ----time_series_profiles.py
>>>> ----save_profiles.py
>>>>
>>>> * Units working with everything in yt (Nathan Goldbaum, Matt Turk, John
>>>> Zuhone)
>>>> ----hse_field.py
>>>> ----simple_off_axis_projection.py
>>>>
>>>> * Miscellany
>>>> ----aligned_cutting_plane.py -- something is wrong with the derived
>>>> quantity AngularMomentumVector() in that it doesn't work fully for gas, but
>>>> seems to work OK for particles.
>>>> ----free_free_field.py--creating a new derived field (Matt Turk, Nathan
>>>> Goldbaum)
>>>> ----simple_slice_with_multiple_fields.py -- vorticity_squared fails as
>>>> a field.
>>>>
>>>>
>>>> This is by no means a requirement to work on this, but it would help to
>>>> take a look at it to see if you can help correct a small bug here and there
>>>> if your name is listed here (or even if it isn't!)
>>>>
>>>> Cameron
>>>>
>>>>
>>>>
>>>> On Tue, Jun 24, 2014 at 6:09 PM, Nathan Goldbaum <nathan12343 at gmail.com
>>>> > wrote:
>>>>
>>>>> Ah - I see that now.
>>>>>
>>>>> Yes, I agree that fixing the cookbook should be a blocker.  FWIW that
>>>>> card is marked as such.  As I said the other day, the blockers are on the
>>>>> yt-3.0 and the documentation board.  We won't do the yt-3.0 release until
>>>>> all the cards marked as blockers are cleared.
>>>>>
>>>>> This is a big task - any help we can get on this from anyone following
>>>>> along would be much appreciated.  There are a lot of little tasks or tasks
>>>>> that can be completed in a couple hours by anyone that has a little with
>>>>> yt-3.0.
>>>>>
>>>>>
>>>>> On Tue, Jun 24, 2014 at 6:04 PM, Cameron Hummels <chummels at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I think all of the cookbook items are blockers to be honest, because
>>>>>> the cookbook recipes should be tests that are seen as failing.
>>>>>>
>>>>>> https://trello.com/c/NJMfoXdl/22-be-able-to-run-the-cookbook
>>>>>>
>>>>>>
>>>>>> On Tue, Jun 24, 2014 at 5:32 PM, Nathan Goldbaum <
>>>>>> nathan12343 at gmail.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Tuesday, June 24, 2014, Cameron Hummels <chummels at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I think there remain some issues in the VR working in 3.0, which I
>>>>>>>> identified on the cookbook post on the trello board for yt-3.0.  For
>>>>>>>> example, I know the overlaying grids and overlaying boundaries does not
>>>>>>>> currently work.
>>>>>>>>
>>>>>>>
>>>>>>> I don't see this on trello.  Can you make a card and mark it as a
>>>>>>> blocker?
>>>>>>>
>>>>>>>
>>>>>>>>  That may be an easy fix, but it's something to keep in mind.  I
>>>>>>>> was going to work on it last week as I was doing the cookbook update, but I
>>>>>>>> figured it was just going to get replaced with the scene interface, so it
>>>>>>>> wasn't worth the time.
>>>>>>>>
>>>>>>>> I guess I'd still like to have all of the API breakage occur in the
>>>>>>>> big jump from 2.x to 3.0, but if people really want to get 3.0 out the door
>>>>>>>> asap, then perhaps that isn't compatible. Personally, I'm +1 on waiting to
>>>>>>>> have all the halo+VR stuff in 3.0 instead of 3.1, but if everyone else
>>>>>>>> wants a 3.0 out sooner, I will not block it.  I think having a super fancy
>>>>>>>> VR and awesome halo interface is one of the big pulls to getting people who
>>>>>>>> have not yet switched to join 3.0 (from both 2.x as well as non-users) even
>>>>>>>> if it takes a few more months, but I may be the minority here.
>>>>>>>>
>>>>>>>> Cameron
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jun 24, 2014 at 10:08 AM, Nathan Goldbaum <
>>>>>>>> nathan12343 at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Ok - I think the script in the issue description is sufficient.
>>>>>>>>>  Let me know if you need something more detailed.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Jun 24, 2014 at 10:07 AM, Matthew Turk <
>>>>>>>>> matthewturk at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> That's the one -- you mentioned it in a blockers email a few days
>>>>>>>>>> ago.
>>>>>>>>>>
>>>>>>>>>> On Tue, Jun 24, 2014 at 12:06 PM, Nathan Goldbaum <
>>>>>>>>>> nathan12343 at gmail.com> wrote:
>>>>>>>>>> > Sorry - not sure which issue you're talking about - this one
>>>>>>>>>> maybe?
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> https://bitbucket.org/yt_analysis/yt/issue/827/enzo-particle-fields-work-differently-than
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > On Tue, Jun 24, 2014 at 10:02 AM, Matthew Turk <
>>>>>>>>>> matthewturk at gmail.com>
>>>>>>>>>> > wrote:
>>>>>>>>>> >>
>>>>>>>>>> >> Related to that, do you have a reproducible script for the
>>>>>>>>>> particle
>>>>>>>>>> >> issue you reported?  If so, could you add that to either an
>>>>>>>>>> issue or a
>>>>>>>>>> >> trello card so I can work on it?
>>>>>>>>>> >>
>>>>>>>>>> >> On Tue, Jun 24, 2014 at 11:58 AM, Nathan Goldbaum <
>>>>>>>>>> nathan12343 at gmail.com>
>>>>>>>>>> >> wrote:
>>>>>>>>>> >> > I'd be +1 on this plan, although we should note that this is
>>>>>>>>>> the plan in
>>>>>>>>>> >> > the
>>>>>>>>>> >> > release announcement.  We may also want to note that there
>>>>>>>>>> are some
>>>>>>>>>> >> > issues
>>>>>>>>>> >> > with volume rendering of oct and particle data at the moment
>>>>>>>>>> (I believe
>>>>>>>>>> >> > that's the case - let me know if I'm wrong there).
>>>>>>>>>> >> >
>>>>>>>>>> >> > I think that leaves analysis modules and documentation as
>>>>>>>>>> the main
>>>>>>>>>> >> > blockers
>>>>>>>>>> >> > for a 3.0 release.
>>>>>>>>>> >> >
>>>>>>>>>> >> > -Nathan
>>>>>>>>>> >> >
>>>>>>>>>> >> >
>>>>>>>>>> >> >
>>>>>>>>>> >> > On Tue, Jun 24, 2014 at 9:53 AM, John ZuHone <
>>>>>>>>>> jzuhone at gmail.com> wrote:
>>>>>>>>>> >> >>
>>>>>>>>>> >> >> +1 on Matt's proposal. -1 on a beta.
>>>>>>>>>> >> >>
>>>>>>>>>> >> >> My worry about a beta release is that it will slow
>>>>>>>>>> adoption, whether
>>>>>>>>>> >> >> rightly or wrongly. I think we agree that we're ready to
>>>>>>>>>> encourage
>>>>>>>>>> >> >> adoption
>>>>>>>>>> >> >> of 3.0.
>>>>>>>>>> >> >>
>>>>>>>>>> >> >> John ZuHone
>>>>>>>>>> >> >> Laboratory for High-Energy Astrophysics
>>>>>>>>>> >> >> NASA/Goddard Space Flight Center
>>>>>>>>>> >> >> 8800 Greenbelt Rd., Mail Code 662
>>>>>>>>>> >> >> Greenbelt, MD 20771
>>>>>>>>>> >> >> (w) 301-286-2531
>>>>>>>>>> >> >> (m) 781-708-5004
>>>>>>>>>> >> >> john.zuhone at nasa.gov
>>>>>>>>>> >> >> jzuhone at gmail.com
>>>>>>>>>> >> >>
>>>>>>>>>> >> >> > On Jun 24, 2014, at 12:38 PM, Matthew Turk <
>>>>>>>>>> matthewturk at gmail.com>
>>>>>>>>>> >> >> > wrote:
>>>>>>>>>> >> >> >
>>>>>>>>>> >> >> > I think Britton covered the halos, but the VR works
>>>>>>>>>> as-is.  As far as
>>>>>>>>>> >> >> > 3.0beta, I'm a bit nervous about that as we want to avoid
>>>>>>>>>> the
>>>>>>>>>> >> >> > situation where we are in beta for 1+ years... I am
>>>>>>>>>> worried about the
>>>>>>>>>> >> >> > perception of a "beta" tag.  Is that overblown?  Would
>>>>>>>>>> calling it
>>>>>>>>>> >> >> > "yt-3.0-2014" work?
>>>>>>>>>> >> >> >
>>>>>>>>>> >> >> >> On Tue, Jun 24, 2014 at 10:32 AM, Nathan Goldbaum
>>>>>>>>>> >> >> >> <nathan12343 at gmail.com> wrote:
>>>>>>>>>> >> >> >> Do the old VR and halo interfaces work?  Not much effort
>>>>>>>>>> has gone
>>>>>>>>>> >> >> >> into
>>>>>>>>>> >> >> >> porting them, I think.
>>>>>>>>>> >> >> >>
>>>>>>>>>> >> >> >>
>>>>>>>>>> >> >> >>> On Tuesday, June 24, 2014, Sam Skillman <
>>>>>>>>>> samskillman at gmail.com>
>>>>>>>>>> >> >> >>> wrote:
>>>>>>>>>> >> >> >>>
>>>>>>>>>> >> >> >>> I'm +1 on this, particularly since I'm at fault for not
>>>>>>>>>> pushing on
>>>>>>>>>> >> >> >>> the
>>>>>>>>>> >> >> >>> VR
>>>>>>>>>> >> >> >>> as much as I'd like to.
>>>>>>>>>> >> >> >>>
>>>>>>>>>> >> >> >>>
>>>>>>>>>> >> >> >>> On Tue, Jun 24, 2014 at 7:44 AM, Matthew Turk
>>>>>>>>>> >> >> >>> <matthewturk at gmail.com>
>>>>>>>>>> >> >> >>> wrote:
>>>>>>>>>> >> >> >>>>
>>>>>>>>>> >> >> >>>> Hi all,
>>>>>>>>>> >> >> >>>>
>>>>>>>>>> >> >> >>>> One thing we really tried to do with 3.0 was break all
>>>>>>>>>> the APIs we
>>>>>>>>>> >> >> >>>> thought we'd need to before release.  This included
>>>>>>>>>> things like
>>>>>>>>>> >> >> >>>> ds/pf,
>>>>>>>>>> >> >> >>>> index/hierarchy, the way data selections were made,
>>>>>>>>>> etc.
>>>>>>>>>> >> >> >>>>
>>>>>>>>>> >> >> >>>> It's starting to become clear that we are approaching
>>>>>>>>>> maturity at
>>>>>>>>>> >> >> >>>> different rates in these initiatives.  I am wondering
>>>>>>>>>> if perhaps
>>>>>>>>>> >> >> >>>> we
>>>>>>>>>> >> >> >>>> should de-couple the release from all of the API
>>>>>>>>>> breakages, and
>>>>>>>>>> >> >> >>>> instead note which interfaces we know are going to
>>>>>>>>>> change in the
>>>>>>>>>> >> >> >>>> future.
>>>>>>>>>> >> >> >>>>
>>>>>>>>>> >> >> >>>> Pragmatically, what this would mean is:
>>>>>>>>>> >> >> >>>>
>>>>>>>>>> >> >> >>>> * Release a 3.0 with the old VR and halo finding
>>>>>>>>>> interfaces
>>>>>>>>>> >> >> >>>> * Release a 3.1 with either the new VR or the new halo
>>>>>>>>>> finding (or
>>>>>>>>>> >> >> >>>> both)
>>>>>>>>>> >> >> >>>> * Do the same for 3.2
>>>>>>>>>> >> >> >>>>
>>>>>>>>>> >> >> >>>> This doesn't fit with the usual "major numbers are
>>>>>>>>>> where APIs
>>>>>>>>>> >> >> >>>> break"
>>>>>>>>>> >> >> >>>> philosophy that comes from semantic versioning, but I
>>>>>>>>>> think from
>>>>>>>>>> >> >> >>>> the
>>>>>>>>>> >> >> >>>> perspective of pragmatism, if we identify those
>>>>>>>>>> sections of the
>>>>>>>>>> >> >> >>>> code
>>>>>>>>>> >> >> >>>> that are *going* to change, and we pitch 3.0 as the
>>>>>>>>>> first part of
>>>>>>>>>> >> >> >>>> a
>>>>>>>>>> >> >> >>>> staged release of totally rewritten infrastructure, we
>>>>>>>>>> can likely
>>>>>>>>>> >> >> >>>> come
>>>>>>>>>> >> >> >>>> out okay.
>>>>>>>>>> >> >> >>>>
>>>>>>>>>> >> >> >>>> I'd like to put this out there for discussion.
>>>>>>>>>> >> >> >>>>
>>>>>>>>>> >> >> >>>> -Matt
>>>>>>>>>> >> >> >>>> _______________________________________________
>>>>>>>>>> >> >> >>>> yt-dev mailing list
>>>>>>>>>> >> >> >>>> yt-dev at lists.spacepope.org
>>>>>>>>>> >> >> >>>>
>>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>>>>>> >> >> >>
>>>>>>>>>> >> >> >> _______________________________________________
>>>>>>>>>> >> >> >> yt-dev mailing list
>>>>>>>>>> >> >> >> yt-dev at lists.spacepope.org
>>>>>>>>>> >> >> >>
>>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>>>>>> >> >> > _______________________________________________
>>>>>>>>>> >> >> > yt-dev mailing list
>>>>>>>>>> >> >> > yt-dev at lists.spacepope.org
>>>>>>>>>> >> >> >
>>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>>>>>> >> >> _______________________________________________
>>>>>>>>>> >> >> yt-dev mailing list
>>>>>>>>>> >> >> yt-dev at lists.spacepope.org
>>>>>>>>>> >> >>
>>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>>>>>> >> >
>>>>>>>>>> >> >
>>>>>>>>>> >> >
>>>>>>>>>> >> > _______________________________________________
>>>>>>>>>> >> > yt-dev mailing list
>>>>>>>>>> >> > yt-dev at lists.spacepope.org
>>>>>>>>>> >> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>>>>>> >> >
>>>>>>>>>> >> _______________________________________________
>>>>>>>>>> >> yt-dev mailing list
>>>>>>>>>> >> yt-dev at lists.spacepope.org
>>>>>>>>>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > _______________________________________________
>>>>>>>>>> > yt-dev mailing list
>>>>>>>>>> > yt-dev at lists.spacepope.org
>>>>>>>>>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>>>>>> >
>>>>>>>>>> _______________________________________________
>>>>>>>>>> yt-dev mailing list
>>>>>>>>>> yt-dev at lists.spacepope.org
>>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> yt-dev mailing list
>>>>>>>>> yt-dev at lists.spacepope.org
>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Cameron Hummels
>>>>>>>> Postdoctoral Researcher
>>>>>>>> Steward Observatory
>>>>>>>> University of Arizona
>>>>>>>> http://chummels.org
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> yt-dev mailing list
>>>>>>> yt-dev at lists.spacepope.org
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Cameron Hummels
>>>>>> Postdoctoral Researcher
>>>>>> Steward Observatory
>>>>>> University of Arizona
>>>>>> http://chummels.org
>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-dev mailing list
>>>>>> yt-dev at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> yt-dev mailing list
>>>>> yt-dev at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Cameron Hummels
>>>> Postdoctoral Researcher
>>>> Steward Observatory
>>>> University of Arizona
>>>> http://chummels.org
>>>>
>>>> _______________________________________________
>>>> yt-dev mailing list
>>>> yt-dev at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>
>>>>
>>>
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>


-- 
Cameron Hummels
Postdoctoral Researcher
Steward Observatory
University of Arizona
http://chummels.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20140625/61cc65ad/attachment.html>


More information about the yt-dev mailing list