[yt-dev] Proposal: deprecate lazy_reader

[Matthew Turk]

Hi all,

Looks like we're leading toward passage.

On Thu, Feb 23, 2012 at 12:02 PM, david collins <antpuncher at gmail.com> wrote:
> +0.98
>
> For what it's worth-- in one script I have wherein I compute profiles
> with a covering grid, I have lazy_reader = false.  I seem to recall
> that it had something to do with covering grids on fields that contain
> spatial derivatives, thus requiring ghost zones.  The line in question
> looks like:
>
> pc.add_profile_object(covering_grid,fields, lazy_reader=False)
>
> d.

Okay, *this* is interesting, and I vaguely recall it.  Let's put
fixing that without the lazy_reader as a use case.  When the merge
request comes for geometry, we'll revisit this.

The merge request will be a ways off, and will touch a lot of things,
so hopefully we can get people on board with testing.

Any chance you could write up something that shows this behavior with
existing spatial derivative fields, so we can add it to the test
suite?

-Matt

>
> On Thu, Feb 23, 2012 at 9:48 AM, Britton Smith <brittonsmith at gmail.com> wrote:
>> +!
>>
>>
>> On Thu, Feb 23, 2012 at 11:46 AM, Cameron Hummels
>> <chummels at astro.columbia.edu> wrote:
>>>
>>> +1
>>>
>>>
>>> On 2/23/12 11:43 AM, Matthew Turk wrote:
>>>>
>>>> Hi all,
>>>>
>>>> Long ago, I added the "lazy_reader" option to profiles.  This then
>>>> showed up also in DerivedQuantities.  In almost all cases used, this
>>>> is set to True, and I believe that the only case when it cannot be is
>>>> when calculating energy.  lazy_reader = True means data is read on
>>>> demand, grid by grid (mediated by preloading for those formats where
>>>> multiple grids might be in a single file) and False means that all of
>>>> the data necessary for the reduction is loaded immediately into
>>>> memory.
>>>>
>>>> I'd like to propose we deprecate it as an option, and force all
>>>> profiles to be "lazy".  I would be implementing this in the geometry
>>>> handling branch.  In the geometry handling branch, I have implemented
>>>> a chunking scheme which is more flexible; in this way subsets of data
>>>> objects can be loaded.  By moving to this scheme, the chunks can
>>>> either be the *entire* data object (i.e., the non-lazy method from
>>>> before) or subsets divided in several different ways.  For example,
>>>> you could do grid-by-grid, you could do gridfile-by-gridfile, or
>>>> whatever type of scheme you like.  I believe that getting rid of lazy
>>>> and moving to just always operating on chunks is the best way to
>>>> accomplish this.
>>>>
>>>> [+-][01] on removing lazy_reader as optional?
>>>>
>>>> -Matt
>>>> _______________________________________________
>>>> yt-dev mailing list
>>>> yt-dev at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>
>>> _______________________________________________
>>> yt-dev mailing list
>>> yt-dev at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>>
>>
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>
>
>
> --
> Sent from my computer.
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org