[yt-dev] Proposal: deprecate lazy_reader

Matthew Turk matthewturk at gmail.com
Thu Feb 23 08:43:05 PST 2012


Hi all,

Long ago, I added the "lazy_reader" option to profiles.  This then
showed up also in DerivedQuantities.  In almost all cases used, this
is set to True, and I believe that the only case when it cannot be is
when calculating energy.  lazy_reader = True means data is read on
demand, grid by grid (mediated by preloading for those formats where
multiple grids might be in a single file) and False means that all of
the data necessary for the reduction is loaded immediately into
memory.

I'd like to propose we deprecate it as an option, and force all
profiles to be "lazy".  I would be implementing this in the geometry
handling branch.  In the geometry handling branch, I have implemented
a chunking scheme which is more flexible; in this way subsets of data
objects can be loaded.  By moving to this scheme, the chunks can
either be the *entire* data object (i.e., the non-lazy method from
before) or subsets divided in several different ways.  For example,
you could do grid-by-grid, you could do gridfile-by-gridfile, or
whatever type of scheme you like.  I believe that getting rid of lazy
and moving to just always operating on chunks is the best way to
accomplish this.

[+-][01] on removing lazy_reader as optional?

-Matt



More information about the yt-dev mailing list