[yt-dev] Merging the unit refactor

[Matthew Turk]

Hi everyone,

I've done a lot of thinking and talking with people about the idea of
merging the units stuff into the mainline yt 3.0 branch.

There are clear advantages to doing this: people who want to use SPH
smoothing would be able to get it from the primary repository, PRs
could be done through that repository, and the access to new things
would be considerably easier.  More public development and review
could happen; while the development already *is* public, it's out of
view in my fork of yt.  This is not productive.

But the development of yt is not the point of yt.  Using yt to enable
scientific discovery is the point of yt.

In many ways, the units refactor will enable more scientific
discovery.  But it's not ready.  There are people using yt-3.0
*already* (prime example: http://nickolas1.com/d3test2/ ) to do really
cool science in ways that they can't with 2.x.  And they're doing this
with a yt that *mostly* works like the 2.x branch, with the same field
names and units and all of that, so the docs *mostly* apply.

The units refactor, if merged in, would pull the rug *completely* out
from under them.  And there's no safety net.  There's a web of YTEPs
and PR comments and notebooks posted to mailing lists, but there's no
place they can go and see, "Hey, this worked before, why isn't it
now?"  And that's not okay.

I've long put off writing documentation, and honestly, I could come up
with lots more reasons to put it off.  But I started on Wednesday
actually writing things down in earnest, and I think that needs to be
the next big push, which I am committed to doing.  Yeah, it's not that
fun always.  Especially since things *are* still changing.  But it's
not fair -- and it is certainly not in the spirit of *extreme empathy*
-- to just change things.

But I also want new development to continue.  And so I want a balance
to be struck.  I'd like to enumerate the items that are necessary for
documentation so that we can merge it in.  I think these are as
follows:

 * All notebooks should be ported to the 3.0 docs and unit-refactor style
 * API documentation has to be able to be compiled
 * At a *bare* minimum, a list of stumbling blocks has to be included
for moving to 3.0.  Britton and I have started on this and made very
good progress.
 * We need a bookmark or tag to be included in the repo *pre*-refactor.
 * Cookbook recipes must work (I think they mostly do now)

Things I don't think we need to do before merging:

 * Completely update 100% of the narrative docs
 * Document how to add smoothing fields, as I believe this API is in flux
 * Describe the underlying methods in great, extensive detail for the
new frontends
 * A full, complete review of the docs like we did in advance of 2.6

As a thought, why don't we treat documentation the way we treat code?
Within the project, it seems we're comfortable committing and
submitting work-in-progress code, but not docs.  In the past, perhaps
this was because the PRs and repos were separate.  They aren't
anymore.

How does this proposal for the merge sound?  Please render an opinion,
as I'd like to have this settled before the early part of next week.

Thanks everyone,

Matt