[Yt-dev] Parallel Hop

[Stephen Skory]

All,


> * Add the HDF5 write functions to the Halo objects.

When it comes to this, I think there should be the choice to write either in 'parallel' to multiple HDF5 files, which is very fast but less convenient, or in 'serial' to one HDF5 file which is very slow, but handy. As an example, writing out HOP data from a 512^3 particle dataset (L7) takes less than 10 minutes in parallel mode, but in serial mode takes over five hours. At least it takes that long on Ranger, and some of that may be due to pytables and very large datasets.

Do we think it's a good idea to include a text file with a parallel write, like for packed AMR enzo, that lists the location of each halo dataset in the .cpu files? The situations I can think of when one would only want a subset of the haloes are when one wants a specific halo or haloes from a physical region of the box. Since the box is spatially decomposed, it may also be useful to record the boundaries of each .cpu file in a simple way.

 _______________________________________________________
sskory at physics.ucsd.edu           o__  Stephen Skory
http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student
________________________________(_)_\(_)_______________