[Yt-dev] Packed AMR hierarchy problem

[Matthew Turk]

> Commenting out __obtain_filenames in trunk doesn't seem to do it for me. My dataset is located where the full pathnames say they are. It's dying inside of _save_data in HierarchyType where it's trying to save the .yt file, due to h5py being unwilling to save an empty array (h5py.h5.ArgsError: Zero sized dimension for non-unlimited dimension). From pdb:
>
> -> arr = myGroup.create_dataset(name,data=array)
> (Pdb) name
> 'DataFields'
> (Pdb) array
> []

Interesting.  Any ideas on how to fix it?  :(

> I've spent the last while trying to get this to work in the hg yt tip, but I think I need some help. I've discovered a couple things in HierarchyType.py that seems wrong. I haven't committed them because I'm not sure I understand what's going on well enough to do that.

As a sanity check, can you do this with the hierarchy-opt branch?  To
make sure I didn't screw anything up during the merge.

> h5py doesn't like directly accessing the file pointer. Adding an intermediate step fixes this hangup:

Commit this.

> These three datasets in the .harrays file are not 1D, they're 3D, so trying to put them in a flatiter array won't work. I might be doing something wrong here, but as it is doesn't seem quite right. Do we want to keep the self...arrays as flatiters?

No, that's not crucial.

> With these fixes I'm hanging in the "for level in xrange(self.max_level+1):" loop in "_initialize_level_stats()" because self.max_level+1 is much much greater than the actual max:

self.max_level is being calculated wrong, and my suspicion is that
it's getting the grid count, not the maximum level.  Can you manually
examine the .harrays file to make sure this is the case?  Looking at
the short output you have below, looks like it might actually be
putting the ParentIDs into the level array.

> Why is MAXLEVEL different than self.max_level?

It's a remnant from the very first piece of code I wrote for yt!  :)
It can probably be replaced with something, but I don't think it's
that big a deal.  It's only used in setting up the level stats, and if
that happens *after* the max_level is set, we can just use that
instead.

-Matt