[yt-users] Covering Grid Dimensions: multiple refinement levels

Nathan Goldbaum nathan12343 at gmail.com
Tue Apr 5 06:55:53 PDT 2016


On Tue, Apr 5, 2016 at 8:49 AM, Carla Bernhardt <carla.j.bernhardt at gmail.com
> wrote:

> Hi Nathan,
>
> Back in November you said "in the upcoming yt 3.3. release and in the
> current development arbitrary_grid can be used for AMR mesh fields like
> ('gas', 'density')." Is this now available? I have come back to this
> problem and still need to find a way to produce unigrid data from a
> specified region (so the left_edge and the right_edge attribute would be
> great) as well as with dimensions that can be processed by a computer.
>

yt 3.3 hasn't been released yet, but you can try installing from the
development branch.

http://yt-project.org/doc/installing.html#installing-yt-using-pip-or-from-source

specifically the steps beginning with "hg clone" at the page I linked to ^


>
> Thanks for your help.
>
> Best,
> Carla
>
> 2015-11-26 15:13 GMT+01:00 Nathan Goldbaum <nathan12343 at gmail.com>:
>
>>
>>
>> On Thu, Nov 26, 2015 at 7:19 AM, Carla Bernhardt <
>> carla.j.bernhardt at gmail.com> wrote:
>>
>>> Dear yt users,
>>>
>>> Thanks again for the help on this topic. To make this work with the
>>> available memory, I want to take a cut region from the data set and put it
>>> into the smoothed_covering_grid(). I know smoothed_covering_grid() uses
>>> a left_edge, but why not a right_edge? Can I cut it after it is made? I
>>> have tried cut_region(), box() and region() but none of these support
>>> smoothed_covering_grid() after application. I have a center and a width
>>> in code_length units.
>>>
>>
>> Both covering_grid and smoothed_covering_grid work the same way, so below
>> I'll only refer to covering_grid, even though this applies equally to both.
>>
>> The job of covering_grid is to create a uniform resolution representation
>> of an AMR simulation, at the same resolution of one of the AMR levels. This
>> explains why you need to specify the AMR level you want to sample at - an
>> AMR simulation has *multiple* spatial resolutions, one for each AMR level,
>> so covering_grid needs a user to tell it which resolution to sample at.
>>
>> The AMR level determines the linear cell spacing of the resulting uniform
>> resolution grid. In general, for a simulation with grid dimensions on level
>> 0 given by "domain_dimensions", and a box size in code units given by
>> box_size, the linear resolution on AMR level "level" is (in python
>> pseudo-code, where both box_size and domain_dimensions act like numpy
>> arrays):
>>
>> dx = box_size / (domain_dimensions * 2**level)
>>
>> In addition, covering_grid needs to know where in the simulation to begin
>> sampling. This is left_edge, the bottom-left hand corner of the region to
>> sample over. In general left_edge needs to be carefully chosen so it falls
>> exactly at a zone boundary at the AMR level you're sampling at.  I believe
>> that if this isn't the case, it will either floor or ceiling left_edge to
>> the nearest zone boundary.
>>
>> You might also expect to be able to to specify a right_edge, however,
>> this would be very misleading and possibly surprising, so it is not done
>> this way. This is because an arbitrary right_edge will most likely not
>> happen to fall an exact integer number of zones away from the left_edge.
>> Instead, covering_grid lets users specify the integer *dimensions* of the
>> resulting grid. To calculate right_edge from the dims, you can do (again
>> interpreting the arrays below as numpy arrays):
>>
>> right_edge = left_edge + dims*dx
>>
>> where dx is from the first equation I gave above.
>>
>> This is definitely not intuitive (I was confused by it at first) but it
>> is a sensible way of doing it once you realize the limitations imposed by
>> the problem. Using covering_grid is "nice" because it's a "natural" way to
>> represent AMR data: you are doing the minimum possible amount of resampling
>> or interpolating, and are using the "real" simulation data on the AMR level
>> you are sampling at.
>>
>> I should also mention that yt also provides arbitrary_grid, which *does*
>> take a left_edge, right_edge, and dims. On the stable branch all extant
>> stable releases of yt, arbitrary_grid can only be used for deposited
>> particle fields. However, in the upcoming yt 3.3. release and in the
>> current development arbitrary_grid can be used for AMR mesh fields like
>> ('gas', 'density'). While arbitrary_grid has an easier to understand API,
>> it is also not as natural a representation of AMR data as covering_grid,
>> since unless left_edge, right_edge, and the dimensions of your
>> arbitrary_grid are carefully chosen, every zone in the result will be
>> affected by resampling or interpolating.
>>
>> Hope that explanation helped!
>>
>> -Nathan
>>
>>
>>>
>>> Cheers,
>>> Carla
>>>
>>> 2015-11-19 18:48 GMT+01:00 Michael Zingale <
>>> michael.zingale at stonybrook.edu>:
>>>
>>>> If I understand this, you are saying it will be a 2048**3 uniform
>>>> grid.  At 8 bytes per #, this is 64 GB of memory to hold the data.
>>>>
>>>> On Thu, Nov 19, 2015 at 12:45 PM, Carla Bernhardt <
>>>> carla.j.bernhardt at gmail.com> wrote:
>>>>
>>>>> Nathan,
>>>>>
>>>>> Dims*2**3 is 256*2^3 or 2048. With the data I am working with, I think
>>>>> this should be on the order of 1 GB. I don't think that should overload the
>>>>> memory... Is it possible there is something incorrectly defined?
>>>>>
>>>>> Thanks,
>>>>> Carla
>>>>>
>>>>>
>>>>> Am Donnerstag, 19. November 2015 schrieb Nathan Goldbaum :
>>>>>
>>>>>> What is dims*2**3? Depending on your data this might be a very large
>>>>>> array that won't fit in memory on your machine.
>>>>>>
>>>>>> You will need to find a machine that has more RAM to do this or
>>>>>> create a covering grid that only covers a subset of the domain.
>>>>>>
>>>>>> On Thursday, November 19, 2015, Carla Bernhardt <
>>>>>> carla.j.bernhardt at gmail.com> wrote:
>>>>>>
>>>>>>> Thanks again for the help. Once I implemented that, I have a
>>>>>>> follow-up problem: Once I have this much larger grid covering, it seems
>>>>>>> possibly too large for yt/python to handle. When I run:
>>>>>>>
>>>>>>> cg=ds.smoothed_covering_grid(3,[0,0,0], dims*2**3)
>>>>>>> data=cg['density']
>>>>>>>
>>>>>>> (where before I had dims*2**1), the variable data does not load and
>>>>>>> gives a few errors <http://pastebin.com/JVhenc4z> which lead me to
>>>>>>> believe that python or yt will not let me use that much space. Can I force
>>>>>>> yt to let me use more space?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Carla
>>>>>>>
>>>>>>> 2015-11-19 16:11 GMT+01:00 Nathan Goldbaum <nathan12343 at gmail.com>:
>>>>>>>
>>>>>>>> Hi Carla,
>>>>>>>>
>>>>>>>> So it looks like ref_factors is only defined in the boxlib
>>>>>>>> frontend, which is why it doesn't work for your Enzo data. Boxlib needs a
>>>>>>>> concept of ref_factors because it allows AMR refinement jumps larger than a
>>>>>>>> factor of two on a single level (i.e. level 2 might be four times higher
>>>>>>>> resolution as level 1). Enzo and many other AMR codes do not allow this,
>>>>>>>> instead only allowing a factor of two increase in linear resolution per AMR
>>>>>>>> level.
>>>>>>>>
>>>>>>>> The dimensions you want to use depend on what fraction of the full
>>>>>>>> domain you want to interpolate to uniform resolution. It looks like you
>>>>>>>> want to create a covering grid covering your full domain, so to create a
>>>>>>>> covering grid at the same resolution as AMR level 3 you would do:
>>>>>>>>
>>>>>>>> all_data_level_3 = ds.covering_grid(level=2, left_edge=[0,0.0,0.0],
>>>>>>>>                                     dims=ds.domain_dimensions *
>>>>>>>> 2**3)
>>>>>>>>
>>>>>>>> This is because each refinement level jump in an Enzo simulation
>>>>>>>> corresponds to a factor of two increase in spatial resolution.
>>>>>>>>
>>>>>>>> Hope that makes sense,
>>>>>>>>
>>>>>>>> Nathan
>>>>>>>>
>>>>>>>> On Thu, Nov 19, 2015 at 8:58 AM, Carla Bernhardt <
>>>>>>>> carla.j.bernhardt at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Thank you for your quick response. However, when I used:
>>>>>>>>>
>>>>>>>>> ref = int(np.product(ds.ref_factors[0:3])),
>>>>>>>>>
>>>>>>>>> I got this error:
>>>>>>>>>
>>>>>>>>> AttributeError: 'EnzoDataset' object has no attribute 'ref_factors'
>>>>>>>>>
>>>>>>>>> Did I misunderstand your suggestion? Or do I need to import
>>>>>>>>> something?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Carla
>>>>>>>>>
>>>>>>>>> 2015-11-19 15:39 GMT+01:00 Michael Zingale <
>>>>>>>>> michael.zingale at stonybrook.edu>:
>>>>>>>>>
>>>>>>>>>> I've done this in the past:
>>>>>>>>>>
>>>>>>>>>> ref = int(np.product(ds.ref_factors[0:max_level]))
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> # allocate for our uniformly-gridded result
>>>>>>>>>>
>>>>>>>>>> dims = ds.domain_dimensions*ref
>>>>>>>>>>
>>>>>>>>>> this will work for a more general case when the jump between
>>>>>>>>>> levels can change as a function of level.
>>>>>>>>>>
>>>>>>>>>> On Thu, Nov 19, 2015 at 9:10 AM, Carla Bernhardt <
>>>>>>>>>> carla.j.bernhardt at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Dear YT Users,
>>>>>>>>>>>
>>>>>>>>>>> To better understand covering_grid (or smoothed_covering_grid),
>>>>>>>>>>> can someone explain what dimensions I should use when I have multiple
>>>>>>>>>>> levels of refinement? If I have 1 level of refinement from AMR data, the
>>>>>>>>>>> dimensions should be the same, I believe, but what if I have 2 or 3 levels
>>>>>>>>>>> of refinement? Should the fixed resolution region then have dimensions of
>>>>>>>>>>> dims*2^2 and dims*3^2 respectively?
>>>>>>>>>>>
>>>>>>>>>>> Here is one example from a tutorial
>>>>>>>>>>> <http://yt-project.org/doc/examining/low_level_inspection.html#examining-grid-data-in-a-fixed-resolution-array>
>>>>>>>>>>> if that helps isolate my question:
>>>>>>>>>>> all_data_level_2 = ds.covering_grid(level=2,
>>>>>>>>>>> left_edge=[0,0.0,0.0],
>>>>>>>>>>>                                       dims=ds.domain_dimensions
>>>>>>>>>>> * 2**2)
>>>>>>>>>>>
>>>>>>>>>>> Thanks in advance,
>>>>>>>>>>> Carla
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> yt-users mailing list
>>>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Michael Zingale
>>>>>>>>>> Associate Professor
>>>>>>>>>>
>>>>>>>>>> Dept. of Physics & Astronomy • Stony Brook University • Stony
>>>>>>>>>> Brook, NY 11794-3800
>>>>>>>>>> *phone*:  631-632-8225
>>>>>>>>>> *e-mail*: Michael.Zingale at stonybrook.edu
>>>>>>>>>> *web*: http://www.astro.sunysb.edu/mzingale
>>>>>>>>>> github: http://github.com/zingale
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> yt-users mailing list
>>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> yt-users mailing list
>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> yt-users mailing list
>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Michael Zingale
>>>> Associate Professor
>>>>
>>>> Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY
>>>> 11794-3800
>>>> *phone*:  631-632-8225
>>>> *e-mail*: Michael.Zingale at stonybrook.edu
>>>> *web*: http://www.astro.sunysb.edu/mzingale
>>>> github: http://github.com/zingale
>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20160405/fa718dc8/attachment.html>


More information about the yt-users mailing list