[yt-users] Announcing: yt 3.0 alpha 3

[Matthew Turk]

Hi all,

Thanks to Colin DeGraf, Chris Moody and Nathan Goldbaum for
identifying and correcting a critical bug in the RAMSES data loader.
This fix has now been pushed and safeguards have been put in place to
prevent a regression.

Best,

Matt

On Wed, Aug 21, 2013 at 2:53 PM, Matthew Turk <matthewturk at gmail.com> wrote:
> Hi everyone,
>
> We're proud to announce the third ALPHA release of yt 3.0.  yt has
> recently transitioned to a time-based release plan (
> https://ytep.readthedocs.org/en/latest/YTEPs/YTEP-0008.html ) and this
> is the third scheduled alpha of 3.0, although we've grossly missed the
> estimated date of July 15.  No date for a "final" release has yet been
> set, but there will likely be several more alphas before that time.
>
> The yt 2.5 codebase, and further updates in the 2.x series, will be
> supported for a considerable amount of time and you do not need to
> upgrade.
>
> = yt 3.0?! =
>
> yt 3.0 represents a new direction forward for yt: getting rid of all
> the underlying assumptions that data needs to be sectioned off into
> nice little grid patches.  This includes supporting Octree codes
> natively (NMSU-ART and RAMSES), eventual support for SPH codes, and
> even opaque data structures where the data is extremely large (ARTIO).
>  We're even planning support for natively handling cylindrical and
> spherical coordinates.
>
> More: http://blog.yt-project.org/post/WhatsUpWith30.html
>
> However, this *is* an alpha release.  Not all of the existing codes
> have been ported to 3.0.
>
> Additionally, this release benefits from the technical and
> non-technical contributions from many new people.  yt is developed in
> the context of a community of contributors, and with the push toward a
> new architecture, we aim to expand that community considerably.  In
> particular, this release has considerably benefited from contributions
> from many new individuals.
>
> = Getting It! =
>
> To try out yt 3.0, you can now pull from the main yt repository,
> update to the yt-3.0 branch, and rebuild your extensions.  Or, if you
> would like to create a new, safely sectioned off environment, simply
> re-run the normal "development" install script after changing the
> variable BRANCH to "yt-3.0".
>
> If you would like to try out yt 3.0 and are having trouble, please
> write to the yt-users mailing list for assistance.
>
> The yt 3.0 install script may also work, which can be obtained by
> executing these commands:
>
>   wget http://hg.yt-project.org/yt/raw/yt-3.0/doc/install_script.sh
>   bash install_script.sh
>
> = What's New? =
>
> A demo notebook demonstrating much new functionality, and including
> the full release notes, can be found here:
>
> http://hub.yt-project.org/nb/88u14f
>
> The bullet-pointed release notes can be found at the end of this
> email, as well.  The main improvements include *considerable* memory
> usage reduction, ARTIO spatial data indexing support, better particle
> support, an overhaul of the selection methods for data objects, and a
> mechanism for on-the-fly definitions of particle types based on
> boolean filters.
>
> Additionally, this release was used in the production of the AGORA
> project flagship paper.  The analysis script used there can be seen at
> this short URL:
>
> http://goo.gl/SwY9Uv
>
> and the paper can be found here: http://arxiv.org/abs/1308.2669
>
> = Reporting Problems =
>
> If you test out yt 3.0 we want to hear if it DID or DID NOT work!
> Feedback is crucial at this time. yt-users and yt-dev are both good
> forums for discussion, asking questions, and reporting problems.  Lots
> of things have changed on the backend, but we have attempted to
> minimize the user-facing changes.
>
> To report a bug please go here:
>
> https://bitbucket.org/yt_analysis/yt/issues/new
>
> Note that you will not receive updates if you are not logged in when
> you create the bug.
>
> = What's Next? =
>
> The next alpha release (3.0a4) will be released sometime this fall,
> but development can be monitored either at
> http://bitbucket.org/yt_analysis/yt-3.0 or in the main yt repository
> under the named branch "yt-3.0".  We hope to focus on generalizing
> Octree support further, adding better non-Cartesian support, and
> supporting generic smoothing kernel definitions.
>
> If you'd like to participate in yt development, please stop by #yt on
> irc.freenode.net ( http://yt-project.org/irc.html ) or yt-dev (
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org ), or
> submit a pull request on BitBucket.
>
> Thanks very much,
>
> Matt, on behalf of the yt development team
>
> Changelog:
>
> (Including 434 changesets from 7 contributors since 3.0a2)
>
>  * Fixes for volume rendering and porting of the Cython kD-tree.  VR
> now works for octree codes, albeit slowly.
> * Major reorganization and simplification of selector code, including
> generic periodicity
> * ARTIO now has spatial data support
> * Major bug fix for FLASH IO.  FLASH works again, following a
> regression in 3.0a2.
> * Merged with mainline yt development, bringing developments from yt
> 2.5.3 and 2.5.4 into 3.0.
> * OWLS (Gadget-HDF5) data now updated to match functionality of other
> Gadget datasets.
> * Creationg of "arbitrary_grid" data selector for flexible particle
> deposition operations.  This enables creation of a grid of arbitrary
> dimensions that can have particles smoothed or deposited within them.
> * Conversion of all CIC operations to particle deposition operations.
> This results in much more flexible particle type selection and speed
> improvements.
> * Refactoring of particle fields to enable simpler creation and
> collection of particle fields.  Particle fields defined for Eulerian
> codes and Lagrangian codes are now interchangeable.
> * New, currently unused parallel ring iteration.  To be explored in
> the future for smoothing kernels.
> * Many improvements for NMSU-ART
> * Fixed a crazy QuadTree deposition bug for projections that showed up
> when projecting octree particle deposition results through a
> restricted data object.
> * Fixed a crazy off-by-one bug for binary Gadget IO.
> * Enormous Octree diet.
>     * Oct leaf nodes now cost 256 bits each, which may eventually be
> reduced to 192 bits.  Previous versions were considerably larger.
>     * Octree traversal is strictly recursive, enabling
> distributed-memory octrees to be implemented.
>     * RAMSES octrees are created by-domain instead of globally.  This
> will enable better parallelism in the future and faster ghost zone
> generation when that is implemented.
>     * Particle octree are now constructed via Z-curve generation.
> This is considerably faster (10-20x) as well as much more memory
> conservative.  This was designed to utilize parallel octree
> construction in the future.
>     * Particle octrees are now indexed by coarse base-level indexing
> of domain regions.  This enables unsorted particle files to be indexed
> for IO as well as spatial selection without mandating that leaf nodes
> are fully-contained within a single file.  Reduction in total oct leaf
> count of ~8x for multi-file particle datasets.
> * YTDataContainer no longer requires .shape and .size
> * Only requested fields are plotted, not translated fields
> * Reduce memory usage of spatially-chunked fields for patch datasets
> * Added arbitrary particle loader, load_particles.
> * Units fix for RAMSES boxlen!=1.0 datasets
> * Particle filtering for dynamically-created particle types, similar
> to derived fields
> * Improve type consistency in TotalQuantity and other derived quantities.
> * Fixed major error with ghost zone generation and uninitialized values
> * Some resiliency for particle datasets that do not specify a domain
> * Bugfix for 1- and 2-D Enzo datasets
> * Improvements to the Tipsy frontend, including auto-detecting parameter files
> * Enable specification of n_ref when creating particle dataset
> * Enable particle deposition to back-act on particle fields
> * Major fix for octree particle counting, resulting in smaller meshes
> * Enable "source" specification for volume rendering
> * Enable initial volume rendering of octree datasets
> * Fixed error with RAMSES C/F ordering of data
> * Many fixes related to field dependency calculation
> * FLASH cylindrical & polar pixelization fix