[yt-dev] proposed change to development process

[Andrew Myers]

I just want to chime in to say I agree with Nathan and Matt re: allowing
more active development on the yt branch. Ideally, I think there should be
two branches (stable and yt) with one head each, with the yt branch being
the "development" version of the code. Most users would be on the stable
branch, and more active users / developers would be on the development
branch. Having multiple heads / bookmarks on the yt branch makes for a more
complicated development process (as evidenced by the fact that none of us
can remember how it works in this very email chain ;) ), which can be
discouraging for new developers. Since creating the experimental bookmark
for the scene refactor, we've already had PRs accidently made and merged
into the wrong head, and it's created more work for Kacper in keeping the
CI working smoothly.

I appreciate the concerns about release deadlines creating pressure to
release under-documented and under-tested code. However, and as others have
stated, with Nathan's backport script, we can now do bugfix releases as
often as needed, which removes some (most? all?) of that pressure.

Additionally, I agree that in practice, most of the serious testing of new
features is going to be done after the code gets merged, and the community
of yt developers / active users who use the development version of the code
get their hands on it. Doing that merge sooner rather than later will
result in an overall faster timeline to stable code.

To be clear, I think Britton's proposal is a definite improvement over the
"forks of forks" model, but I think an even simpler development process
with only one head on the main yt branch would be more productive in the
long run.

On Thu, Sep 10, 2015 at 1:23 PM, Matthew Turk <matthewturk at gmail.com> wrote:

> Hi Britton,
>
> I was going to reply, broadly in support of everything you suggested,
> until I saw the other emails.  It looks like I missed my opportunity.
>
> In general, I would like to see more experimental development in the main
> yt repo; I think with the backporting script, as you suggest, we now have a
> good reason for people to use "stable" instead of the main development
> branch.
>
> So I guess "multiple heads" even with multiple bookmarks is off the table
> now, if I read the rest of the thread correctly?  If so, can we figure out
> a way to allow experimental stuff into "yt" and then move most folks onto
> "stable"?
>
> On Thu, Sep 10, 2015 at 1:52 PM, Britton Smith <brittonsmith at gmail.com>
> wrote:
>
>> Hi everyone,
>>
>> I had some ideas for improving the yt development process that I
>> wanted to run by everyone.  This can be discussed further at our
>> upcoming team meeting and if people are in favor, I will issue a pull
>> request to the relevant YTEP.
>>
>> STATEMENT OF PROBLEM
>> Currently, development proceeds roughly as follows.  The two main
>> active branches within the central yt repository are *yt* and *stable*.
>> The tip of *stable* is the latest release and the *yt* branch is the de
>> facto "development version" of the code.  Until recently, we have not
>> been very good at regularly scheduled minor releases and so the *stable*
>> branch sits for quite some time with many bugs that are fixed within
>> the development branch.  This effectively makes *stable* unusable and
>> pushes most users to the *yt* branch.
>>
>> When new features are developed, pull requests are issued to the
>> single head of the *yt* branch.  Because this is the version most people
>> are actually using, the current policy is to not allow PR with new
>> functionality to be accepted until they are 100% ready (full
>> functionality, tests, docs, etc).  As we have already seen, this makes
>> collaborative development very cumbersome, as it requires people to
>> create forks of the fork from which the PR originates.  They then must
>> issue PRs to that fork after which time the original PR is updated.
>> The current volume render refactor is the perfect example of this.
>>
>> PROPOSED SOLUTION
>> Before I lay out the proposed solution, I want to list a number of
>> recent developments that I think will make this possible:
>> 1. Nathan's new script for backporting changes now keeps *stable* and
>> *yt*
>>    synced on bugfixes.
>> 2. We have returned to doing minor releases containing only bugfixes,
>>    thanks again to Nathan's hard work.  This and point 1 means that
>>    users are once again safe to be on *stable*, and now *should* be there
>>    most of the time.
>> 3. Bitbucket now supports bookmarks, meaning that PRs can be issued to
>>    specific bookmarks instead of to branches or heads named only by the
>>    changeset hash.
>> 4. The weekly PR triage hangouts are making it easier to process PRs
>>    and also providing a place to strategize getting larger PRs
>>    accepted.  Thanks to Hilary for keeping this going.
>>
>> With the above in mind, I propose the following:
>> 1. Create a "development" bookmark to sit at the tip of the *yt*
>>    branch.  All PRs containing relatively small new features are
>>    issued to this.  The requirements for acceptance remain the same:
>>    PRs accepted to "development" must contain all intended
>>    functionality and be fully documented.  This allows the
>>    "development" bookmark to be defined explicitly as everything that
>>    will be included in the next major release.
>> 2. Large new features should have a corresponding YTEP that has been
>>    accepted.  After the YTEP has been accepted, a PR should be issued
>>    to the *yt* branch.  After some initial discussion, this PR is pulled
>>    into the main yt repo with a bookmark named after the feature.
>>    Once this has happened, developers can now issue new PRs
>>    specifically to this bookmark.  This is effectively what we have
>>    now with the volume render work in the "experimental" bookmark,
>>    only we would rename the bookmark to something like "vr-refactor".
>>    As with PRs issued directly to "development", only after the new
>>    feature is 100% ready shall it be merged into the "development"
>>    bookmark.
>> 3. We continue to make use of the PR triage hangouts to establish when
>>    large features are ready to be merged.
>>
>> I believe this will have the following benefits:
>> 1. Large, new features can be developed collaboratively without the
>>    need for forks of forks of forks.
>> 2. New, underdevelopment features are more accessible to the larger
>>    community by simply updating to named bookmarks from the main repo
>>    (no need for "just pull these changes from my fork").
>> 3. The "development" branch is preserved as a place only for
>>    ready-to-be-released features (i.e., polished and documented).
>>
>>
>> All told, this is really just a small tweak on our current process.
>> Please comment with any thoughts, or even a +/-1 if your feelings can
>> be summed up thusly.
>>
>> Britton
>>
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20150910/71cb292a/attachment.html>