Yeah, this is exactly that problem. When reading the hierarchy file, all the grid filenames are put into a list, and then re-associated with the grids later. If you have a dm-only dataset, grids with no particles do not have any filename entry in the hierarchy file, thus you get a list of file and a list of grids of different length. The problem is that __obtain_filenames assumes the lists are the same size, and just lays them down next to each other. So, for grids later in the hierarchy than any grid without particles, they are associated with the wrong filename.<br>
<br>Commenting out __obtain_filenames will work as long as the paths in the hierarchy are correct, since it's that routine that works out the correct paths to the data. I didn't commit the fix for this reason, as I've found it pretty valuable not to have to change all those paths in the hierarchy file every time I copy data somewhere else.<br>
<br>On a related note, how would you guys feel if enzo was changed such that the hierarchy file and such only write the relative path to the filename? So, for restarts, it would read in the value of GlobalDir and then, when reading the hierarchy file and other places where file paths are given, it would then say:<br>
filename = GlobalDir + <file path on line of hierarchy file><br>Personally, I find it a bit redundant that we keep track of GlobalDir and write full paths to files.<br><br>Britton<br><br><div class="gmail_quote">On Wed, Nov 18, 2009 at 9:55 PM, Matthew Turk <span dir="ltr"><<a href="mailto:matthewturk@gmail.com">matthewturk@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Hi Stephen,<br>
<br>
This is the same problem Britton reported. cpu_map is unused, but it<br>
gets created anyway. The current trunk hierarchy is not great.<br>
<br>
Can you try commenting out the call to __obtain_filenames inside<br>
populate_hierarchy, and also try with the current mercurial tip, to<br>
see if either or both of those things fix the problem?<br>
<br>
-Matt<br>
<div><div></div><div class="h5"><br>
On Wed, Nov 18, 2009 at 8:52 PM, Stephen Skory <<a href="mailto:stephenskory@yahoo.com">stephenskory@yahoo.com</a>> wrote:<br>
> Hi (Matt, mainly),<br>
><br>
> I have a packed AMR, dm-only dataset that is giving me some odd issues with the hierarchy. The grids are not being referenced to the correct AMR HDF5 file. These commands:<br>
><br>
> from yt.mods import *<br>
> pf = load('RedshiftOutput0005', data_style="enzo_packed_3d")<br>
> pf.h.cpu_map['/mirage/sskory/reddead-survey-amr/RD0005/RedshiftOutput0005.cpu0004'][:10]<br>
><br>
> give:<br>
> [EnzoGrid_0005,<br>
> EnzoGrid_1867,<br>
> EnzoGrid_1868,<br>
> EnzoGrid_1869,<br>
> EnzoGrid_1870,<br>
> EnzoGrid_1871,<br>
> EnzoGrid_1872,<br>
> EnzoGrid_1873,<br>
> EnzoGrid_1874,<br>
> EnzoGrid_1875]<br>
><br>
> This shows that grid 1867 is owned by cpu 4. But here's what the .hierarchy file says about Grid 1867, that it's owned by cpu0005. A manual inspection shows that it is indeed inside of the cpu0005 HDF5 file.<br>
><br>
> Grid = 1867<br>
> Task = 5<br>
> GridRank = 3<br>
> GridDimension = 12 12 12<br>
> GridStartIndex = 3 3 3<br>
> GridEndIndex = 8 8 8<br>
> GridLeftEdge = 0.5078125 0.521484375 0.337890625<br>
> GridRightEdge = 0.513671875 0.52734375 0.34375<br>
> Time = 646.75066015177<br>
> SubgridsAreStatic = 0<br>
> NumberOfBaryonFields = 0<br>
> NumberOfParticles = 96<br>
> ParticleFileName = /mirage/sskory/reddead-survey-amr/RD0005/RedshiftOutput0005.cpu0005<br>
> GravityBoundaryType = 2<br>
> Pointer: Grid[1867]->NextGridThisLevel = 0<br>
> Pointer: Grid[1867]->NextGridNextLevel = 0<br>
> Pointer: Grid[5]->NextGridNextLevel = 1868<br>
><br>
> I'm using the latest trunk, and the files should be world-readable. Any ideas why this is happening?<br>
><br>
> Thanks!<br>
><br>
> _______________________________________________________<br>
> <a href="mailto:sskory@physics.ucsd.edu">sskory@physics.ucsd.edu</a> o__ Stephen Skory<br>
> <a href="http://physics.ucsd.edu/%7Esskory/" target="_blank">http://physics.ucsd.edu/~sskory/</a> _.>/ _Graduate Student<br>
> ________________________________(_)_\(_)_______________<br>
><br>
> _______________________________________________<br>
> Yt-dev mailing list<br>
> <a href="mailto:Yt-dev@lists.spacepope.org">Yt-dev@lists.spacepope.org</a><br>
> <a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
><br>
_______________________________________________<br>
Yt-dev mailing list<br>
<a href="mailto:Yt-dev@lists.spacepope.org">Yt-dev@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
</div></div></blockquote></div><br>