[Yt-dev] Packed AMR hierarchy problem

[Britton Smith]

Yeah, this is exactly that problem.  When reading the hierarchy file, all
the grid filenames are put into a list, and then re-associated with the
grids later.  If you have a dm-only dataset, grids with no particles do not
have any filename entry in the hierarchy file, thus you get a list of file
and a list of grids of different length.  The problem is that
__obtain_filenames assumes the lists are the same size, and just lays them
down next to each other.  So, for grids later in the hierarchy than any grid
without particles, they are associated with the wrong filename.

Commenting out __obtain_filenames will work as long as the paths in the
hierarchy are correct, since it's that routine that works out the correct
paths to the data.  I didn't commit the fix for this reason, as I've found
it pretty valuable not to have to change all those paths in the hierarchy
file every time I copy data somewhere else.

On a related note, how would you guys feel if enzo was changed such that the
hierarchy file and such only write the relative path to the filename?  So,
for restarts, it would read in the value of GlobalDir and then, when reading
the hierarchy file and other places where file paths are given, it would
then say:
filename = GlobalDir + <file path on line of hierarchy file>
Personally, I find it a bit redundant that we keep track of GlobalDir and
write full paths to files.

Britton

On Wed, Nov 18, 2009 at 9:55 PM, Matthew Turk <matthewturk at gmail.com> wrote:

> Hi Stephen,
>
> This is the same problem Britton reported.  cpu_map is unused, but it
> gets created anyway.  The current trunk hierarchy is not great.
>
> Can you try commenting out the call to __obtain_filenames inside
> populate_hierarchy, and also try with the current mercurial tip, to
> see if either or both of those things fix the problem?
>
> -Matt
>
> On Wed, Nov 18, 2009 at 8:52 PM, Stephen Skory <stephenskory at yahoo.com>
> wrote:
> > Hi (Matt, mainly),
> >
> > I have a packed AMR, dm-only dataset that is giving me some odd issues
> with the hierarchy. The grids are not being referenced to the correct AMR
> HDF5 file. These commands:
> >
> > from yt.mods import *
> > pf = load('RedshiftOutput0005', data_style="enzo_packed_3d")
> >
> pf.h.cpu_map['/mirage/sskory/reddead-survey-amr/RD0005/RedshiftOutput0005.cpu0004'][:10]
> >
> > give:
> > [EnzoGrid_0005,
> >  EnzoGrid_1867,
> >  EnzoGrid_1868,
> >  EnzoGrid_1869,
> >  EnzoGrid_1870,
> >  EnzoGrid_1871,
> >  EnzoGrid_1872,
> >  EnzoGrid_1873,
> >  EnzoGrid_1874,
> >  EnzoGrid_1875]
> >
> > This shows that grid 1867 is owned by cpu 4. But here's what the
> .hierarchy file says about Grid 1867, that it's owned by cpu0005. A manual
> inspection shows that it is indeed inside of the cpu0005 HDF5 file.
> >
> > Grid = 1867
> > Task              = 5
> > GridRank          = 3
> > GridDimension     = 12 12 12
> > GridStartIndex    = 3 3 3
> > GridEndIndex      = 8 8 8
> > GridLeftEdge      = 0.5078125 0.521484375 0.337890625
> > GridRightEdge     = 0.513671875 0.52734375 0.34375
> > Time              = 646.75066015177
> > SubgridsAreStatic = 0
> > NumberOfBaryonFields = 0
> > NumberOfParticles   = 96
> > ParticleFileName =
> /mirage/sskory/reddead-survey-amr/RD0005/RedshiftOutput0005.cpu0005
> > GravityBoundaryType = 2
> > Pointer: Grid[1867]->NextGridThisLevel = 0
> > Pointer: Grid[1867]->NextGridNextLevel = 0
> > Pointer: Grid[5]->NextGridNextLevel = 1868
> >
> > I'm using the latest trunk, and the files should be world-readable. Any
> ideas why this is happening?
> >
> > Thanks!
> >
> >  _______________________________________________________
> > sskory at physics.ucsd.edu           o__  Stephen Skory
> > http://physics.ucsd.edu/~sskory/ <http://physics.ucsd.edu/%7Esskory/>_.>/ _Graduate Student
> > ________________________________(_)_\(_)_______________
> >
> > _______________________________________________
> > Yt-dev mailing list
> > Yt-dev at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >
> _______________________________________________
> Yt-dev mailing list
> Yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20091119/40be917f/attachment.htm>