[yt-dev] proposed change to development process
Matthew Turk
matthewturk at gmail.com
Thu Sep 10 12:19:58 PDT 2015
On Thu, Sep 10, 2015 at 2:18 PM, Cameron Hummels <chummels at gmail.com> wrote:
> In general, I like Britton's idea as it fosters more group development as
> opposed to the problems of forks of forks that many of us experience in
> trying to develop unmerged features like VR-refactor and ytdata frontend.
>
> That said, there are some issues to be aware of. By default, I think when
> you pull from the repository, hg defaults to updating to the most recent
> commit (ie the tip), which may not be in your active bookmark. This means
> that if someone just committed to the "ytdata frontend" head/bookmark, and
> I pull from the main repository expecting to get the main "development"
> bookmark, I'm going to be put on the wrong version of the code. This can
> be confusing unless we educate everyone to always make sure after they pull
> to update to the "development" bookmark or whatever.
>
As a quick note, this isn't true -- it updates to the "@" bookmark.
I have more thoughts on the other emails as well, but wanted to take this
quick opportunity to be pedantic and annoying.
>
> I'm dubious of the idea of merging more work-in-progress code into the
> main head of the dev branch as Nathan suggests. As he points out, the
> success of this model is entirely dependent on the author/community
> "finishing" code that goes into the dev branch before it gets tagged as a
> stable release. All this does is simply kick the can down the road as to
> when code gets "finished" (i.e. documented, tested, polished, the parts no
> one wants to do). By delaying this process until there is a deadline (the
> stable release) it separates it from when the main development went on, and
> makes it less likely for authors to remember what was going on in the
> code. In the past, this results in code kinda getting pushed through, just
> to not hold up the release any longer, and it ends up being unnecessarily
> acrimonious. I'm in favor or the separate heads/bookmarks model that
> Britton suggests, but not merging those heads into the main dev bookmark
> until it is "finished".
>
> Cameron
>
>
> On Thu, Sep 10, 2015 at 12:01 PM, Nathan Goldbaum <nathan12343 at gmail.com>
> wrote:
>
>> Just a first initial reaction: I don't think introducing even more heads
>> to the yet branch is a good idea, even if they are marked by bookmarks. In
>> fact I'd prefer to only have one head on the yt branch right now.
>>
>> In addition, bitbucket's support for bookmark's in the pull request
>> interface amounts only to labeling entries in the source and destination
>> dropdown menu, meaning that once a pull request is merged, someone would
>> need to manually update the corresponding bookmark to the correct head.
>> This is both error prone and easy to skip, so I predict that if we *did*
>> start doing this, bookmarks would wuickly go out of sync with heads,
>> leaving a bunch of un-bookmarked heads.
>>
>> Was this prompted by your work on the reloadable datasets PR? Without
>> something like this do you expect it to sit in limbo for a long time?
>>
>> I think the solution we need is to jot be afraid to pull in more
>> "experimental" work-in-progress stuff to the yt branch. Of course that
>> would need to come with assurances from the author or community
>> members that they're committed to "finishing" the code before the next
>> major release. That way people will be able to test new code more easily
>> (they'll get it without updating explicitly to it) and we'll catch
>> breakages to unrelated code and wordflows more quickly.
>>
>> Nathan
>>
>>
>> On Thursday, September 10, 2015, Britton Smith <brittonsmith at gmail.com>
>> wrote:
>>
>>> Hi everyone,
>>>
>>> I had some ideas for improving the yt development process that I
>>> wanted to run by everyone. This can be discussed further at our
>>> upcoming team meeting and if people are in favor, I will issue a pull
>>> request to the relevant YTEP.
>>>
>>> STATEMENT OF PROBLEM
>>> Currently, development proceeds roughly as follows. The two main
>>> active branches within the central yt repository are *yt* and *stable*.
>>> The tip of *stable* is the latest release and the *yt* branch is the de
>>> facto "development version" of the code. Until recently, we have not
>>> been very good at regularly scheduled minor releases and so the *stable*
>>> branch sits for quite some time with many bugs that are fixed within
>>> the development branch. This effectively makes *stable* unusable and
>>> pushes most users to the *yt* branch.
>>>
>>> When new features are developed, pull requests are issued to the
>>> single head of the *yt* branch. Because this is the version most people
>>> are actually using, the current policy is to not allow PR with new
>>> functionality to be accepted until they are 100% ready (full
>>> functionality, tests, docs, etc). As we have already seen, this makes
>>> collaborative development very cumbersome, as it requires people to
>>> create forks of the fork from which the PR originates. They then must
>>> issue PRs to that fork after which time the original PR is updated.
>>> The current volume render refactor is the perfect example of this.
>>>
>>> PROPOSED SOLUTION
>>> Before I lay out the proposed solution, I want to list a number of
>>> recent developments that I think will make this possible:
>>> 1. Nathan's new script for backporting changes now keeps *stable* and
>>> *yt*
>>> synced on bugfixes.
>>> 2. We have returned to doing minor releases containing only bugfixes,
>>> thanks again to Nathan's hard work. This and point 1 means that
>>> users are once again safe to be on *stable*, and now *should* be
>>> there
>>> most of the time.
>>> 3. Bitbucket now supports bookmarks, meaning that PRs can be issued to
>>> specific bookmarks instead of to branches or heads named only by the
>>> changeset hash.
>>> 4. The weekly PR triage hangouts are making it easier to process PRs
>>> and also providing a place to strategize getting larger PRs
>>> accepted. Thanks to Hilary for keeping this going.
>>>
>>> With the above in mind, I propose the following:
>>> 1. Create a "development" bookmark to sit at the tip of the *yt*
>>> branch. All PRs containing relatively small new features are
>>> issued to this. The requirements for acceptance remain the same:
>>> PRs accepted to "development" must contain all intended
>>> functionality and be fully documented. This allows the
>>> "development" bookmark to be defined explicitly as everything that
>>> will be included in the next major release.
>>> 2. Large new features should have a corresponding YTEP that has been
>>> accepted. After the YTEP has been accepted, a PR should be issued
>>> to the *yt* branch. After some initial discussion, this PR is pulled
>>> into the main yt repo with a bookmark named after the feature.
>>> Once this has happened, developers can now issue new PRs
>>> specifically to this bookmark. This is effectively what we have
>>> now with the volume render work in the "experimental" bookmark,
>>> only we would rename the bookmark to something like "vr-refactor".
>>> As with PRs issued directly to "development", only after the new
>>> feature is 100% ready shall it be merged into the "development"
>>> bookmark.
>>> 3. We continue to make use of the PR triage hangouts to establish when
>>> large features are ready to be merged.
>>>
>>> I believe this will have the following benefits:
>>> 1. Large, new features can be developed collaboratively without the
>>> need for forks of forks of forks.
>>> 2. New, underdevelopment features are more accessible to the larger
>>> community by simply updating to named bookmarks from the main repo
>>> (no need for "just pull these changes from my fork").
>>> 3. The "development" branch is preserved as a place only for
>>> ready-to-be-released features (i.e., polished and documented).
>>>
>>>
>>> All told, this is really just a small tweak on our current process.
>>> Please comment with any thoughts, or even a +/-1 if your feelings can
>>> be summed up thusly.
>>>
>>> Britton
>>>
>>
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>>
>
>
> --
> Cameron Hummels
> NSF Postdoctoral Fellow
> Department of Astronomy
> California Institute of Technology
> http://chummels.org
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20150910/de0bd63d/attachment.html>
More information about the yt-dev
mailing list