[Yt-dev] Release 1.5

[Matthew Turk]

Hi guys,

I'd like to take a moment to sketch out my proposal for releasing
yt-1.5.  We all run off trunk, but right now, 1.0 is the version
included with Enzo.  Furthermore, the documentation has not been
substantially updated since 1.0, so almost all of the stuff we have
been doing and working on for the past year (yes, *year*) has not been
advertised or included.  I'd like to compile some of the things that
have happened, most of which have been visible in the ticket tracker,
but not all.

http://yt.enzotools.org/query?status=closed&group=resolution&milestone=1.5

There are a couple things that don't show up quite so visibly:

 * Fully parallel slices, projections, cutting planes, profiles,
quantities (although scaling will be a target of 2.0)
 * Mostly parallel HOP (scaling is a major problem here)
 * FoF halo finding
 * Object storage and serialization
 * Major performance improvements to the clump finder (factor ~5)
 * Generalized domain sizes
 * Generalized field info containers (which need some work for 2.0)
 * Dark Matter-only simulations
 * 1D and 2D simulations
 * Better IO for HDF5 sets
 * Support for the Orion AMR code
 * Spherical re-gridding
 * Halo profiler (ticket 95 needs some love)
 * Light cone generator
 * Callback interface improved
 * Several new callbacks
 * New data objects -- ortho and non-ortho rays, limited ray-tracing
 * Fixed resolution buffers
 * Spectral integrator for CLOUDY data
 * Substantially better interactive interface
 * Performance improvements basically everywhere
 * Command-line interface to *many* common tasks
 * Isolated plot handling, independent of PlotCollections

I'm pretty sure there's a lot more I'm forgetting -- but the
improvements have been numerous.  I think the biggest thing to
emphasize here would be that yt works in parallel.  Scaling is not
ideal.  That's going to be a task for a later date, I do believe, and
for now I think it's important to push out the improvements --
frankly, I'm a little embarrassed people may be using yt-1.0 at this
point, because so much work has gone on in the trunk to improve
basically every aspect of the code.  (And yes, our audience is small,
by almost all metrics: but I think this particular set of improvements
will help to increase it.)  I think this release is much more
compelling than the last, and I think we all have something to be
proud of.

We still have to finish up these tickets:

http://yt.enzotools.org/report/12

Most or all of these are my repsonsibility; I'd like some help with
#95, though.  Additionally, if anybody wants to volunteer to look at
(even if just attempting to replicate and getting proper tracebacks)
#189, #191 and #197, I'd really appreciate it.  #208 might get bumped
to 2.0, since I'm the only one using fido anymore; I'd like to bring
it back in some way some day, but right now it's not a priority.

Now, for the meat of the remaining process: the documentation.  This
will be primarily my responsibility, but I will need help from other
people.  Specifically, I need someone to volunteer to read and edit
what I write.  Additionally, I would like to submit a request that
sections of prose be written, in the style of a tutorial, by the
people who really are the experts on different sections of the code.
I've noticed a substantial uptick in feelings of ownership of various
sections, and I'm really happy about that.  I think we really need
some semi-tutorial style sections written by the people who feel they
know best certain sections of the code that are really valuable and
useful.  Here's my proposed set:

 * Halo finding: Stephen
 * Halo profiler: Britton
 * Clump finder: Britton
 * Light Cone Generator: Britton
 * Interactive Usage: Jeff

Now, I know you are all very busy -- me too! -- so if this doesn't
work for you, it's all good, just let me know.  :)  I'll handle all
the format conversions and integration and so forth.  The current
documentation repo (has not been put in SVN, but will be once it's
working better) is here:  http://hg.enzotools.org/yt-doc/   .

Furthermore, there's a page set up for comments on the existing documentation:

http://yt.enzotools.org/wiki/Plans/DocEnhancements

Please, if you have any complaints about existing documentation,
making them there will ensure that while I go through the process of
editing and rewriting, I will ensure they are taken into account.
Additionally, while I am willing to take the burden of documentation,
I would really love it if somebody else would help out.  So, let me
know if you are interested.

Okay guys, thanks so much for trudging through this email.  Any
thoughts on any of this?  Did I forget any big features?  Anybody feel
like helping with extra documentation?  Anything else that needs to go
in?  We've all put a lot of work into this stuff, and I think we have
a great chance of making a big splash with a new release.

Thanks guys, and talk to you soon,

-Matt