[yt-users] Covering Grid Dimensions: multiple refinement levels

Carla Bernhardt carla.j.bernhardt at gmail.com
Thu Nov 26 06:44:14 PST 2015 

Nathan,

Thank you for your explanation. I had looked at arbitrary_grid, but I want
to confirm that the data is the same as my AMR data and I thought this
would present a problem. You mention that I "might also expect to be able
to specify a right_edge", but where is this right_edge implemented? I tried
just putting in right_edge=left_edge + dims*dx in the
smoothed_covering_grid command, but it throws an error that there is no
right_edge (which I knew from reading the documentation, I was just hoping
I would get lucky).

Thanks,
Carla

2015-11-26 15:13 GMT+01:00 Nathan Goldbaum <nathan12343 at gmail.com>:

>
>
> On Thu, Nov 26, 2015 at 7:19 AM, Carla Bernhardt <
> carla.j.bernhardt at gmail.com> wrote:
>
>> Dear yt users,
>>
>> Thanks again for the help on this topic. To make this work with the
>> available memory, I want to take a cut region from the data set and put it
>> into the smoothed_covering_grid(). I know smoothed_covering_grid() uses
>> a left_edge, but why not a right_edge? Can I cut it after it is made? I
>> have tried cut_region(), box() and region() but none of these support
>> smoothed_covering_grid() after application. I have a center and a width
>> in code_length units.
>>
>
> Both covering_grid and smoothed_covering_grid work the same way, so below
> I'll only refer to covering_grid, even though this applies equally to both.
>
> The job of covering_grid is to create a uniform resolution representation
> of an AMR simulation, at the same resolution of one of the AMR levels. This
> explains why you need to specify the AMR level you want to sample at - an
> AMR simulation has *multiple* spatial resolutions, one for each AMR level,
> so covering_grid needs a user to tell it which resolution to sample at.
>
> The AMR level determines the linear cell spacing of the resulting uniform
> resolution grid. In general, for a simulation with grid dimensions on level
> 0 given by "domain_dimensions", and a box size in code units given by
> box_size, the linear resolution on AMR level "level" is (in python
> pseudo-code, where both box_size and domain_dimensions act like numpy
> arrays):
>
> dx = box_size / (domain_dimensions * 2**level)
>
> In addition, covering_grid needs to know where in the simulation to begin
> sampling. This is left_edge, the bottom-left hand corner of the region to
> sample over. In general left_edge needs to be carefully chosen so it falls
> exactly at a zone boundary at the AMR level you're sampling at.  I believe
> that if this isn't the case, it will either floor or ceiling left_edge to
> the nearest zone boundary.
>
> You might also expect to be able to to specify a right_edge, however, this
> would be very misleading and possibly surprising, so it is not done this
> way. This is because an arbitrary right_edge will most likely not happen to
> fall an exact integer number of zones away from the left_edge. Instead,
> covering_grid lets users specify the integer *dimensions* of the resulting
> grid. To calculate right_edge from the dims, you can do (again interpreting
> the arrays below as numpy arrays):
>
> right_edge = left_edge + dims*dx
>
> where dx is from the first equation I gave above.
>
> This is definitely not intuitive (I was confused by it at first) but it is
> a sensible way of doing it once you realize the limitations imposed by the
> problem. Using covering_grid is "nice" because it's a "natural" way to
> represent AMR data: you are doing the minimum possible amount of resampling
> or interpolating, and are using the "real" simulation data on the AMR level
> you are sampling at.
>
> I should also mention that yt also provides arbitrary_grid, which *does*
> take a left_edge, right_edge, and dims. On the stable branch all extant
> stable releases of yt, arbitrary_grid can only be used for deposited
> particle fields. However, in the upcoming yt 3.3. release and in the
> current development arbitrary_grid can be used for AMR mesh fields like
> ('gas', 'density'). While arbitrary_grid has an easier to understand API,
> it is also not as natural a representation of AMR data as covering_grid,
> since unless left_edge, right_edge, and the dimensions of your
> arbitrary_grid are carefully chosen, every zone in the result will be
> affected by resampling or interpolating.
>
> Hope that explanation helped!
>
> -Nathan
>
>
>>
>> Cheers,
>> Carla
>>
>> 2015-11-19 18:48 GMT+01:00 Michael Zingale <
>> michael.zingale at stonybrook.edu>:
>>
>>> If I understand this, you are saying it will be a 2048**3 uniform grid.
>>> At 8 bytes per #, this is 64 GB of memory to hold the data.
>>>
>>> On Thu, Nov 19, 2015 at 12:45 PM, Carla Bernhardt <
>>> carla.j.bernhardt at gmail.com> wrote:
>>>
>>>> Nathan,
>>>>
>>>> Dims*2**3 is 256*2^3 or 2048. With the data I am working with, I think
>>>> this should be on the order of 1 GB. I don't think that should overload the
>>>> memory... Is it possible there is something incorrectly defined?
>>>>
>>>> Thanks,
>>>> Carla
>>>>
>>>>
>>>> Am Donnerstag, 19. November 2015 schrieb Nathan Goldbaum :
>>>>
>>>>> What is dims*2**3? Depending on your data this might be a very large
>>>>> array that won't fit in memory on your machine.
>>>>>
>>>>> You will need to find a machine that has more RAM to do this or create
>>>>> a covering grid that only covers a subset of the domain.
>>>>>
>>>>> On Thursday, November 19, 2015, Carla Bernhardt <
>>>>> carla.j.bernhardt at gmail.com> wrote:
>>>>>
>>>>>> Thanks again for the help. Once I implemented that, I have a
>>>>>> follow-up problem: Once I have this much larger grid covering, it seems
>>>>>> possibly too large for yt/python to handle. When I run:
>>>>>>
>>>>>> cg=ds.smoothed_covering_grid(3,[0,0,0], dims*2**3)
>>>>>> data=cg['density']
>>>>>>
>>>>>> (where before I had dims*2**1), the variable data does not load and
>>>>>> gives a few errors <http://pastebin.com/JVhenc4z> which lead me to
>>>>>> believe that python or yt will not let me use that much space. Can I force
>>>>>> yt to let me use more space?
>>>>>>
>>>>>> Thanks,
>>>>>> Carla
>>>>>>
>>>>>> 2015-11-19 16:11 GMT+01:00 Nathan Goldbaum <nathan12343 at gmail.com>:
>>>>>>
>>>>>>> Hi Carla,
>>>>>>>
>>>>>>> So it looks like ref_factors is only defined in the boxlib frontend,
>>>>>>> which is why it doesn't work for your Enzo data. Boxlib needs a concept of
>>>>>>> ref_factors because it allows AMR refinement jumps larger than a factor of
>>>>>>> two on a single level (i.e. level 2 might be four times higher resolution
>>>>>>> as level 1). Enzo and many other AMR codes do not allow this, instead only
>>>>>>> allowing a factor of two increase in linear resolution per AMR level.
>>>>>>>
>>>>>>> The dimensions you want to use depend on what fraction of the full
>>>>>>> domain you want to interpolate to uniform resolution. It looks like you
>>>>>>> want to create a covering grid covering your full domain, so to create a
>>>>>>> covering grid at the same resolution as AMR level 3 you would do:
>>>>>>>
>>>>>>> all_data_level_3 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0],
>>>>>>>                                     dims=ds.domain_dimensions * 2**3)
>>>>>>>
>>>>>>> This is because each refinement level jump in an Enzo simulation
>>>>>>> corresponds to a factor of two increase in spatial resolution.
>>>>>>>
>>>>>>> Hope that makes sense,
>>>>>>>
>>>>>>> Nathan
>>>>>>>
>>>>>>> On Thu, Nov 19, 2015 at 8:58 AM, Carla Bernhardt <
>>>>>>> carla.j.bernhardt at gmail.com> wrote:
>>>>>>>
>>>>>>>> Thank you for your quick response. However, when I used:
>>>>>>>>
>>>>>>>> ref = int(np.product(ds.ref_factors[0:3])),
>>>>>>>>
>>>>>>>> I got this error:
>>>>>>>>
>>>>>>>> AttributeError: 'EnzoDataset' object has no attribute 'ref_factors'
>>>>>>>>
>>>>>>>> Did I misunderstand your suggestion? Or do I need to import
>>>>>>>> something?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Carla
>>>>>>>>
>>>>>>>> 2015-11-19 15:39 GMT+01:00 Michael Zingale <
>>>>>>>> michael.zingale at stonybrook.edu>:
>>>>>>>>
>>>>>>>>> I've done this in the past:
>>>>>>>>>
>>>>>>>>> ref = int(np.product(ds.ref_factors[0:max_level]))
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> # allocate for our uniformly-gridded result
>>>>>>>>>
>>>>>>>>> dims = ds.domain_dimensions*ref
>>>>>>>>>
>>>>>>>>> this will work for a more general case when the jump between
>>>>>>>>> levels can change as a function of level.
>>>>>>>>>
>>>>>>>>> On Thu, Nov 19, 2015 at 9:10 AM, Carla Bernhardt <
>>>>>>>>> carla.j.bernhardt at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Dear YT Users,
>>>>>>>>>>
>>>>>>>>>> To better understand covering_grid (or smoothed_covering_grid),
>>>>>>>>>> can someone explain what dimensions I should use when I have multiple
>>>>>>>>>> levels of refinement? If I have 1 level of refinement from AMR data, the
>>>>>>>>>> dimensions should be the same, I believe, but what if I have 2 or 3 levels
>>>>>>>>>> of refinement? Should the fixed resolution region then have dimensions of
>>>>>>>>>> dims*2^2 and dims*3^2 respectively?
>>>>>>>>>>
>>>>>>>>>> Here is one example from a tutorial
>>>>>>>>>> <http://yt-project.org/doc/examining/low_level_inspection.html#examining-grid-data-in-a-fixed-resolution-array>
>>>>>>>>>> if that helps isolate my question:
>>>>>>>>>> all_data_level_2 = ds.covering_grid(level=2,
>>>>>>>>>> left_edge=[0,0.0,0.0],
>>>>>>>>>>                                       dims=ds.domain_dimensions *
>>>>>>>>>> 2**2)
>>>>>>>>>>
>>>>>>>>>> Thanks in advance,
>>>>>>>>>> Carla
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> yt-users mailing list
>>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Michael Zingale
>>>>>>>>> Associate Professor
>>>>>>>>>
>>>>>>>>> Dept. of Physics & Astronomy • Stony Brook University • Stony
>>>>>>>>> Brook, NY 11794-3800
>>>>>>>>> *phone*:  631-632-8225
>>>>>>>>> *e-mail*: Michael.Zingale at stonybrook.edu
>>>>>>>>> *web*: http://www.astro.sunysb.edu/mzingale
>>>>>>>>> github: http://github.com/zingale
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> yt-users mailing list
>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> yt-users mailing list
>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> yt-users mailing list
>>>>>>> yt-users at lists.spacepope.org
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>
>>>>>>>
>>>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>
>>>
>>> --
>>> Michael Zingale
>>> Associate Professor
>>>
>>> Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY
>>> 11794-3800
>>> *phone*:  631-632-8225
>>> *e-mail*: Michael.Zingale at stonybrook.edu
>>> *web*: http://www.astro.sunysb.edu/mzingale
>>> github: http://github.com/zingale
>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20151126/39d1c60f/attachment.htm>

More information about the yt-users mailing list