[yt-dev] proposed change to development process
Nathan Goldbaum
nathan12343 at gmail.com
Thu Sep 10 12:50:13 PDT 2015
On Thu, Sep 10, 2015 at 2:19 PM, Matthew Turk <matthewturk at gmail.com> wrote:
>
>
> On Thu, Sep 10, 2015 at 2:18 PM, Cameron Hummels <chummels at gmail.com>
> wrote:
>
>> In general, I like Britton's idea as it fosters more group development as
>> opposed to the problems of forks of forks that many of us experience in
>> trying to develop unmerged features like VR-refactor and ytdata frontend.
>>
>> That said, there are some issues to be aware of. By default, I think
>> when you pull from the repository, hg defaults to updating to the most
>> recent commit (ie the tip), which may not be in your active bookmark. This
>> means that if someone just committed to the "ytdata frontend"
>> head/bookmark, and I pull from the main repository expecting to get the
>> main "development" bookmark, I'm going to be put on the wrong version of
>> the code. This can be confusing unless we educate everyone to always make
>> sure after they pull to update to the "development" bookmark or whatever.
>>
>
> As a quick note, this isn't true -- it updates to the "@" bookmark.
>
> I have more thoughts on the other emails as well, but wanted to take this
> quick opportunity to be pedantic and annoying.
>
To be pedantically pedantic, this isn't true, and I think Cameron's was
correct. The "@" bookmark is only considered when cloning a repository.
Bare "hg update" will update to the tip of the named branch, even if that
tip is on a different topological branch.
>
>
>>
>> I'm dubious of the idea of merging more work-in-progress code into the
>> main head of the dev branch as Nathan suggests. As he points out, the
>> success of this model is entirely dependent on the author/community
>> "finishing" code that goes into the dev branch before it gets tagged as a
>> stable release. All this does is simply kick the can down the road as to
>> when code gets "finished" (i.e. documented, tested, polished, the parts no
>> one wants to do). By delaying this process until there is a deadline (the
>> stable release) it separates it from when the main development went on, and
>> makes it less likely for authors to remember what was going on in the
>> code. In the past, this results in code kinda getting pushed through, just
>> to not hold up the release any longer, and it ends up being unnecessarily
>> acrimonious. I'm in favor or the separate heads/bookmarks model that
>> Britton suggests, but not merging those heads into the main dev bookmark
>> until it is "finished".
>>
>> Cameron
>>
>>
>> On Thu, Sep 10, 2015 at 12:01 PM, Nathan Goldbaum <nathan12343 at gmail.com>
>> wrote:
>>
>>> Just a first initial reaction: I don't think introducing even more heads
>>> to the yet branch is a good idea, even if they are marked by bookmarks. In
>>> fact I'd prefer to only have one head on the yt branch right now.
>>>
>>> In addition, bitbucket's support for bookmark's in the pull request
>>> interface amounts only to labeling entries in the source and destination
>>> dropdown menu, meaning that once a pull request is merged, someone would
>>> need to manually update the corresponding bookmark to the correct head.
>>> This is both error prone and easy to skip, so I predict that if we *did*
>>> start doing this, bookmarks would wuickly go out of sync with heads,
>>> leaving a bunch of un-bookmarked heads.
>>>
>>> Was this prompted by your work on the reloadable datasets PR? Without
>>> something like this do you expect it to sit in limbo for a long time?
>>>
>>> I think the solution we need is to jot be afraid to pull in more
>>> "experimental" work-in-progress stuff to the yt branch. Of course that
>>> would need to come with assurances from the author or community
>>> members that they're committed to "finishing" the code before the next
>>> major release. That way people will be able to test new code more
>>> easily (they'll get it without updating explicitly to it) and we'll catch
>>> breakages to unrelated code and wordflows more quickly.
>>>
>>> Nathan
>>>
>>>
>>> On Thursday, September 10, 2015, Britton Smith <brittonsmith at gmail.com>
>>> wrote:
>>>
>>>> Hi everyone,
>>>>
>>>> I had some ideas for improving the yt development process that I
>>>> wanted to run by everyone. This can be discussed further at our
>>>> upcoming team meeting and if people are in favor, I will issue a pull
>>>> request to the relevant YTEP.
>>>>
>>>> STATEMENT OF PROBLEM
>>>> Currently, development proceeds roughly as follows. The two main
>>>> active branches within the central yt repository are *yt* and *stable*.
>>>> The tip of *stable* is the latest release and the *yt* branch is the de
>>>> facto "development version" of the code. Until recently, we have not
>>>> been very good at regularly scheduled minor releases and so the
>>>> *stable*
>>>> branch sits for quite some time with many bugs that are fixed within
>>>> the development branch. This effectively makes *stable* unusable and
>>>> pushes most users to the *yt* branch.
>>>>
>>>> When new features are developed, pull requests are issued to the
>>>> single head of the *yt* branch. Because this is the version most
>>>> people
>>>> are actually using, the current policy is to not allow PR with new
>>>> functionality to be accepted until they are 100% ready (full
>>>> functionality, tests, docs, etc). As we have already seen, this makes
>>>> collaborative development very cumbersome, as it requires people to
>>>> create forks of the fork from which the PR originates. They then must
>>>> issue PRs to that fork after which time the original PR is updated.
>>>> The current volume render refactor is the perfect example of this.
>>>>
>>>> PROPOSED SOLUTION
>>>> Before I lay out the proposed solution, I want to list a number of
>>>> recent developments that I think will make this possible:
>>>> 1. Nathan's new script for backporting changes now keeps *stable* and
>>>> *yt*
>>>> synced on bugfixes.
>>>> 2. We have returned to doing minor releases containing only bugfixes,
>>>> thanks again to Nathan's hard work. This and point 1 means that
>>>> users are once again safe to be on *stable*, and now *should* be
>>>> there
>>>> most of the time.
>>>> 3. Bitbucket now supports bookmarks, meaning that PRs can be issued to
>>>> specific bookmarks instead of to branches or heads named only by the
>>>> changeset hash.
>>>> 4. The weekly PR triage hangouts are making it easier to process PRs
>>>> and also providing a place to strategize getting larger PRs
>>>> accepted. Thanks to Hilary for keeping this going.
>>>>
>>>> With the above in mind, I propose the following:
>>>> 1. Create a "development" bookmark to sit at the tip of the *yt*
>>>> branch. All PRs containing relatively small new features are
>>>> issued to this. The requirements for acceptance remain the same:
>>>> PRs accepted to "development" must contain all intended
>>>> functionality and be fully documented. This allows the
>>>> "development" bookmark to be defined explicitly as everything that
>>>> will be included in the next major release.
>>>> 2. Large new features should have a corresponding YTEP that has been
>>>> accepted. After the YTEP has been accepted, a PR should be issued
>>>> to the *yt* branch. After some initial discussion, this PR is
>>>> pulled
>>>> into the main yt repo with a bookmark named after the feature.
>>>> Once this has happened, developers can now issue new PRs
>>>> specifically to this bookmark. This is effectively what we have
>>>> now with the volume render work in the "experimental" bookmark,
>>>> only we would rename the bookmark to something like "vr-refactor".
>>>> As with PRs issued directly to "development", only after the new
>>>> feature is 100% ready shall it be merged into the "development"
>>>> bookmark.
>>>> 3. We continue to make use of the PR triage hangouts to establish when
>>>> large features are ready to be merged.
>>>>
>>>> I believe this will have the following benefits:
>>>> 1. Large, new features can be developed collaboratively without the
>>>> need for forks of forks of forks.
>>>> 2. New, underdevelopment features are more accessible to the larger
>>>> community by simply updating to named bookmarks from the main repo
>>>> (no need for "just pull these changes from my fork").
>>>> 3. The "development" branch is preserved as a place only for
>>>> ready-to-be-released features (i.e., polished and documented).
>>>>
>>>>
>>>> All told, this is really just a small tweak on our current process.
>>>> Please comment with any thoughts, or even a +/-1 if your feelings can
>>>> be summed up thusly.
>>>>
>>>> Britton
>>>>
>>>
>>> _______________________________________________
>>> yt-dev mailing list
>>> yt-dev at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>
>>>
>>
>>
>> --
>> Cameron Hummels
>> NSF Postdoctoral Fellow
>> Department of Astronomy
>> California Institute of Technology
>> http://chummels.org
>>
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20150910/cbad8f74/attachment.html>
More information about the yt-dev
mailing list