[yt-dev] proposed change to development process

Thu Sep 10 12:50:13 PDT 2015

On Thu, Sep 10, 2015 at 2:19 PM, Matthew Turk <matthewturk at gmail.com> wrote:

>
>
> On Thu, Sep 10, 2015 at 2:18 PM, Cameron Hummels <chummels at gmail.com>
> wrote:
>
>> In general, I like Britton's idea as it fosters more group development as
>> opposed to the problems of forks of forks that many of us experience in
>> trying to develop unmerged features like VR-refactor and ytdata frontend.
>>
>> That said, there are some issues to be aware of.  By default, I think
>> when you pull from the repository, hg defaults to updating to the most
>> recent commit (ie the tip), which may not be in your active bookmark.  This
>> means that if someone just committed to the "ytdata frontend"
>> head/bookmark, and I pull from the main repository expecting to get the
>> main "development" bookmark, I'm going to be put on the wrong version of
>> the code.  This can be confusing unless we educate everyone to always make
>> sure after they pull to update to the "development" bookmark or whatever.
>>
>
> As a quick note, this isn't true -- it updates to the "@" bookmark.
>
> I have more thoughts on the other emails as well, but wanted to take this
> quick opportunity to be pedantic and annoying.
>

To be pedantically pedantic, this isn't true, and I think Cameron's was
correct. The "@" bookmark is only considered when cloning a repository.
Bare "hg update" will update to the tip of the named branch, even if that
tip is on a different topological branch.

>
>
>>
>> I'm dubious of the idea of merging more work-in-progress code into the
>> main head of the dev branch as Nathan suggests.  As he points out, the
>> success of this model is entirely dependent on the author/community
>> "finishing" code that goes into the dev branch before it gets tagged as a
>> stable release.  All this does is simply kick the can down the road as to
>> when code gets "finished" (i.e. documented, tested, polished, the parts no
>> one wants to do).  By delaying this process until there is a deadline (the
>> stable release) it separates it from when the main development went on, and
>> makes it less likely for authors to remember what was going on in the
>> code.  In the past, this results in code kinda getting pushed through, just
>> to not hold up the release any longer, and it ends up being unnecessarily
>> acrimonious.  I'm in favor or the separate heads/bookmarks model that
>> Britton suggests, but not merging those heads into the main dev bookmark
>> until it is "finished".
>>
>> Cameron
>>
>>
>> On Thu, Sep 10, 2015 at 12:01 PM, Nathan Goldbaum <nathan12343 at gmail.com>
>> wrote:
>>
>>> Just a first initial reaction: I don't think introducing even more heads
>>> to the yet branch is a good idea, even if they are marked by bookmarks. In
>>> fact I'd prefer to only have one head on the yt branch right now.
>>>
>>> In addition, bitbucket's support for bookmark's in the pull request
>>> interface amounts only to labeling entries in the source and destination
>>> dropdown menu, meaning that once a pull request is merged, someone would
>>> need to manually update the corresponding bookmark to the correct head.
>>> This is both error prone and easy to skip, so I predict that if we *did*
>>> start doing this, bookmarks would wuickly go out of sync with heads,
>>> leaving a bunch of un-bookmarked heads.
>>>
>>> Was this prompted by your work on the reloadable datasets PR? Without
>>> something like this do you expect it to sit in limbo for a long time?
>>>
>>> I think the solution we need is to jot be afraid to pull in more
>>> "experimental" work-in-progress stuff to the yt branch. Of course that
>>> would need to come with assurances from the author or community
>>> members that they're committed to "finishing" the code before the next
>>> major release. That way people will be able to test new code more
>>> easily (they'll get it without updating explicitly to it) and we'll catch
>>> breakages to unrelated code and wordflows more quickly.
>>>
>>> Nathan
>>>
>>>
>>> On Thursday, September 10, 2015, Britton Smith <brittonsmith at gmail.com>
>>> wrote:
>>>
>>>> Hi everyone,
>>>>
>>>> I had some ideas for improving the yt development process that I
>>>> wanted to run by everyone.  This can be discussed further at our
>>>> upcoming team meeting and if people are in favor, I will issue a pull
>>>> request to the relevant YTEP.
>>>>
>>>> STATEMENT OF PROBLEM
>>>> Currently, development proceeds roughly as follows.  The two main
>>>> active branches within the central yt repository are *yt* and *stable*.
>>>> The tip of *stable* is the latest release and the *yt* branch is the de
>>>> facto "development version" of the code.  Until recently, we have not
>>>> been very good at regularly scheduled minor releases and so the
>>>> *stable*
>>>> branch sits for quite some time with many bugs that are fixed within
>>>> the development branch.  This effectively makes *stable* unusable and
>>>> pushes most users to the *yt* branch.
>>>>
>>>> When new features are developed, pull requests are issued to the
>>>> single head of the *yt* branch.  Because this is the version most
>>>> people
>>>> are actually using, the current policy is to not allow PR with new
>>>> functionality to be accepted until they are 100% ready (full
>>>> functionality, tests, docs, etc).  As we have already seen, this makes
>>>> collaborative development very cumbersome, as it requires people to
>>>> create forks of the fork from which the PR originates.  They then must
>>>> issue PRs to that fork after which time the original PR is updated.
>>>> The current volume render refactor is the perfect example of this.
>>>>
>>>> PROPOSED SOLUTION
>>>> Before I lay out the proposed solution, I want to list a number of
>>>> recent developments that I think will make this possible:
>>>> 1. Nathan's new script for backporting changes now keeps *stable* and
>>>> *yt*
>>>>    synced on bugfixes.
>>>> 2. We have returned to doing minor releases containing only bugfixes,
>>>>    thanks again to Nathan's hard work.  This and point 1 means that
>>>>    users are once again safe to be on *stable*, and now *should* be
>>>> there
>>>>    most of the time.
>>>> 3. Bitbucket now supports bookmarks, meaning that PRs can be issued to
>>>>    specific bookmarks instead of to branches or heads named only by the
>>>>    changeset hash.
>>>> 4. The weekly PR triage hangouts are making it easier to process PRs
>>>>    and also providing a place to strategize getting larger PRs
>>>>    accepted.  Thanks to Hilary for keeping this going.
>>>>
>>>> With the above in mind, I propose the following:
>>>> 1. Create a "development" bookmark to sit at the tip of the *yt*
>>>>    branch.  All PRs containing relatively small new features are
>>>>    issued to this.  The requirements for acceptance remain the same:
>>>>    PRs accepted to "development" must contain all intended
>>>>    functionality and be fully documented.  This allows the
>>>>    "development" bookmark to be defined explicitly as everything that
>>>>    will be included in the next major release.
>>>> 2. Large new features should have a corresponding YTEP that has been
>>>>    accepted.  After the YTEP has been accepted, a PR should be issued
>>>>    to the *yt* branch.  After some initial discussion, this PR is
>>>> pulled
>>>>    into the main yt repo with a bookmark named after the feature.
>>>>    Once this has happened, developers can now issue new PRs
>>>>    specifically to this bookmark.  This is effectively what we have
>>>>    now with the volume render work in the "experimental" bookmark,
>>>>    only we would rename the bookmark to something like "vr-refactor".
>>>>    As with PRs issued directly to "development", only after the new
>>>>    feature is 100% ready shall it be merged into the "development"
>>>>    bookmark.
>>>> 3. We continue to make use of the PR triage hangouts to establish when
>>>>    large features are ready to be merged.
>>>>
>>>> I believe this will have the following benefits:
>>>> 1. Large, new features can be developed collaboratively without the
>>>>    need for forks of forks of forks.
>>>> 2. New, underdevelopment features are more accessible to the larger
>>>>    community by simply updating to named bookmarks from the main repo
>>>>    (no need for "just pull these changes from my fork").
>>>> 3. The "development" branch is preserved as a place only for
>>>>    ready-to-be-released features (i.e., polished and documented).
>>>>
>>>>
>>>> All told, this is really just a small tweak on our current process.
>>>> Please comment with any thoughts, or even a +/-1 if your feelings can
>>>> be summed up thusly.
>>>>
>>>> Britton
>>>>
>>>
>>> _______________________________________________
>>> yt-dev mailing list
>>> yt-dev at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>
>>>
>>
>>
>> --
>> Cameron Hummels
>> NSF Postdoctoral Fellow
>> Department of Astronomy
>> California Institute of Technology
>> http://chummels.org
>>
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20150910/cbad8f74/attachment.htm>