[yt-dev] New Workflow

[Matthew Turk]

Hi all,

Over the last couple days, in case you hadn't noticed, we've had a lot
of emails going out about fixing up the workflow, developing a suite
of tests, and handling tracking of changes, bugs, planning, etc etc.
I think these are coming to fruition, and I wanted to write with a
quick summary of where we're at.

1) Changes, except for obvious bug fixes, should be mediated by pull
requests.  This is where we have been for a while.
2) New functionality needs to be accompanied by unit tests.  Right now
we're still working on coming up with a solution for visualization
tasks, but everything else is being covered as we speak.  This can be
something as *simple* as making sure that an API doesn't change, or
doesn't raise an exception, or DOES raise an exception.  All of these
things are valid to do.  We want not only to verify that we're doing
operations correctly, but also that APIs -- especially deep APIs that
other functionality relies on! -- don't change without warning or
reason.  Cameron and I chatted about this today in the context of
moving making functionality; part of the tests should be to make sure
that the APIs such functionality would rely on don't change, or behave
the same way.
3) Frontends will be tested in the near future.  This will be what we
currently call "Answer Testing" but will be more curated and
structured.  Once I have finished up the changes necessary for this
I'll be writing with more.
4) New functionality or big bug fixes should be mediated by the issue
tracker, even if that issue is filled out *after* the bug fix is
submitted.

--

As an example:

Nathan issued this PR:
https://bitbucket.org/yt_analysis/yt/pull-request/291/vector-coordinate-transformations/diff
 .  He then added unit tests during the PR process.  After it was
accepted, it was added to the Shining Panda buildbot (immediately!)
and run:

https://jenkins.shiningpanda-ci.com/yt-project/

Everything came up sunny on the buildbot side, but if it had failed,
it would have reported it to the yt-svn email list:

http://lists.spacepope.org/listinfo.cgi/yt-svn-spacepope.org

After it was accepted, he filled out this ticket:

https://bitbucket.org/yt_analysis/yt/issue/448/vector-coordinate-transformations

Now when we prepare the 2.5 release notes, we will see this in the
list of resolved issues.

--

If you're adding a new test, let's say to
yt/utilities/tests/test_some_utility.py , you can run:

nosetests utilities/tests/test_some_utility.py

(without that leading yt/ on the filename; not sure why this is!) to
run just the tests in that file.  If you want to run all of them:

nosetests -w yt -e "answer_testing"
-or-
python2.7 setup.py nosetests

Soon we'll get rid of the exclude option, when we can convert answer
testing over.

--

So to sum up: add tests, do things with PRs, and fill out issues.  I
think once we become more comfortable as a community writing and
relying on tests, we can start making the big changes we want to while
retaining reliability.

Thanks everybody for bearing with this as we solidify this workflow.
Any thoughts or comments on this process?

-Matt

PS We should be hearing in a week or two about JIRA, but I'm really
happy about the Shining Panda stuff.  Please do let me know if you'd
like your repository added.  3.0 will be going on the list shortly.