[yt-dev] proposed change to development process

Thu Sep 10 12:19:58 PDT 2015

On Thu, Sep 10, 2015 at 2:18 PM, Cameron Hummels <chummels at gmail.com> wrote:

> In general, I like Britton's idea as it fosters more group development as
> opposed to the problems of forks of forks that many of us experience in
> trying to develop unmerged features like VR-refactor and ytdata frontend.
>
> That said, there are some issues to be aware of.  By default, I think when
> you pull from the repository, hg defaults to updating to the most recent
> commit (ie the tip), which may not be in your active bookmark.  This means
> that if someone just committed to the "ytdata frontend" head/bookmark, and
> I pull from the main repository expecting to get the main "development"
> bookmark, I'm going to be put on the wrong version of the code.  This can
> be confusing unless we educate everyone to always make sure after they pull
> to update to the "development" bookmark or whatever.
>

As a quick note, this isn't true -- it updates to the "@" bookmark.

I have more thoughts on the other emails as well, but wanted to take this
quick opportunity to be pedantic and annoying.

>
> I'm dubious of the idea of merging more work-in-progress code into the
> main head of the dev branch as Nathan suggests.  As he points out, the
> success of this model is entirely dependent on the author/community
> "finishing" code that goes into the dev branch before it gets tagged as a
> stable release.  All this does is simply kick the can down the road as to
> when code gets "finished" (i.e. documented, tested, polished, the parts no
> one wants to do).  By delaying this process until there is a deadline (the
> stable release) it separates it from when the main development went on, and
> makes it less likely for authors to remember what was going on in the
> code.  In the past, this results in code kinda getting pushed through, just
> to not hold up the release any longer, and it ends up being unnecessarily
> acrimonious.  I'm in favor or the separate heads/bookmarks model that
> Britton suggests, but not merging those heads into the main dev bookmark
> until it is "finished".
>
> Cameron
>
>
> On Thu, Sep 10, 2015 at 12:01 PM, Nathan Goldbaum <nathan12343 at gmail.com>
> wrote:
>
>> Just a first initial reaction: I don't think introducing even more heads
>> to the yet branch is a good idea, even if they are marked by bookmarks. In
>> fact I'd prefer to only have one head on the yt branch right now.
>>
>> In addition, bitbucket's support for bookmark's in the pull request
>> interface amounts only to labeling entries in the source and destination
>> dropdown menu, meaning that once a pull request is merged, someone would
>> need to manually update the corresponding bookmark to the correct head.
>> This is both error prone and easy to skip, so I predict that if we *did*
>> start doing this, bookmarks would wuickly go out of sync with heads,
>> leaving a bunch of un-bookmarked heads.
>>
>> Was this prompted by your work on the reloadable datasets PR? Without
>> something like this do you expect it to sit in limbo for a long time?
>>
>> I think the solution we need is to jot be afraid to pull in more
>> "experimental" work-in-progress stuff to the yt branch. Of course that
>> would need to come with assurances from the author or community
>> members that they're committed to "finishing" the code before the next
>> major release. That way people will be able to test new code more easily
>> (they'll get it without updating explicitly to it) and we'll catch
>> breakages to unrelated code and wordflows more quickly.
>>
>> Nathan
>>
>>
>> On Thursday, September 10, 2015, Britton Smith <brittonsmith at gmail.com>
>> wrote:
>>
>>> Hi everyone,
>>>
>>> I had some ideas for improving the yt development process that I
>>> wanted to run by everyone.  This can be discussed further at our
>>> upcoming team meeting and if people are in favor, I will issue a pull
>>> request to the relevant YTEP.
>>>
>>> STATEMENT OF PROBLEM
>>> Currently, development proceeds roughly as follows.  The two main
>>> active branches within the central yt repository are *yt* and *stable*.
>>> The tip of *stable* is the latest release and the *yt* branch is the de
>>> facto "development version" of the code.  Until recently, we have not
>>> been very good at regularly scheduled minor releases and so the *stable*
>>> branch sits for quite some time with many bugs that are fixed within
>>> the development branch.  This effectively makes *stable* unusable and
>>> pushes most users to the *yt* branch.
>>>
>>> When new features are developed, pull requests are issued to the
>>> single head of the *yt* branch.  Because this is the version most people
>>> are actually using, the current policy is to not allow PR with new
>>> functionality to be accepted until they are 100% ready (full
>>> functionality, tests, docs, etc).  As we have already seen, this makes
>>> collaborative development very cumbersome, as it requires people to
>>> create forks of the fork from which the PR originates.  They then must
>>> issue PRs to that fork after which time the original PR is updated.
>>> The current volume render refactor is the perfect example of this.
>>>
>>> PROPOSED SOLUTION
>>> Before I lay out the proposed solution, I want to list a number of
>>> recent developments that I think will make this possible:
>>> 1. Nathan's new script for backporting changes now keeps *stable* and
>>> *yt*
>>>    synced on bugfixes.
>>> 2. We have returned to doing minor releases containing only bugfixes,
>>>    thanks again to Nathan's hard work.  This and point 1 means that
>>>    users are once again safe to be on *stable*, and now *should* be
>>> there
>>>    most of the time.
>>> 3. Bitbucket now supports bookmarks, meaning that PRs can be issued to
>>>    specific bookmarks instead of to branches or heads named only by the
>>>    changeset hash.
>>> 4. The weekly PR triage hangouts are making it easier to process PRs
>>>    and also providing a place to strategize getting larger PRs
>>>    accepted.  Thanks to Hilary for keeping this going.
>>>
>>> With the above in mind, I propose the following:
>>> 1. Create a "development" bookmark to sit at the tip of the *yt*
>>>    branch.  All PRs containing relatively small new features are
>>>    issued to this.  The requirements for acceptance remain the same:
>>>    PRs accepted to "development" must contain all intended
>>>    functionality and be fully documented.  This allows the
>>>    "development" bookmark to be defined explicitly as everything that
>>>    will be included in the next major release.
>>> 2. Large new features should have a corresponding YTEP that has been
>>>    accepted.  After the YTEP has been accepted, a PR should be issued
>>>    to the *yt* branch.  After some initial discussion, this PR is pulled
>>>    into the main yt repo with a bookmark named after the feature.
>>>    Once this has happened, developers can now issue new PRs
>>>    specifically to this bookmark.  This is effectively what we have
>>>    now with the volume render work in the "experimental" bookmark,
>>>    only we would rename the bookmark to something like "vr-refactor".
>>>    As with PRs issued directly to "development", only after the new
>>>    feature is 100% ready shall it be merged into the "development"
>>>    bookmark.
>>> 3. We continue to make use of the PR triage hangouts to establish when
>>>    large features are ready to be merged.
>>>
>>> I believe this will have the following benefits:
>>> 1. Large, new features can be developed collaboratively without the
>>>    need for forks of forks of forks.
>>> 2. New, underdevelopment features are more accessible to the larger
>>>    community by simply updating to named bookmarks from the main repo
>>>    (no need for "just pull these changes from my fork").
>>> 3. The "development" branch is preserved as a place only for
>>>    ready-to-be-released features (i.e., polished and documented).
>>>
>>>
>>> All told, this is really just a small tweak on our current process.
>>> Please comment with any thoughts, or even a +/-1 if your feelings can
>>> be summed up thusly.
>>>
>>> Britton
>>>
>>
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>>
>
>
> --
> Cameron Hummels
> NSF Postdoctoral Fellow
> Department of Astronomy
> California Institute of Technology
> http://chummels.org
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20150910/de0bd63d/attachment.htm>