[yt-dev] parallelHOP in yt-3

[Josh Moloney]

I've recently been trying to get parallelHOP integrated into the current
version of yt-3. (I need halo finding with star particles which I think
rules out Rockstar, and I eventually need to work with grids up to 1536^3
which would be problematic with regular HOP.)

I currently have it working properly for single core runs, and it produces
output that matches the yt-2.x version. It also runs to completion with
mpi, and produces mostly sensible output.

However, when run on multiple cores it has problems with the halos near the
region boundaries. In particular it produces a large number (~10% of total)
of typically small "bad halos" which have 0 values for the halo maximum
density/location (i.e. [0, 0, 0, 0]), and at least one negative value for
the center of mass location (due to incorrect handling of periodicity). In
addition, some of the "real" halos near the region boundaries have fewer
particles in parallel runs. Comparing halo particle lists between serial
and mpi runs shows that some of these missing particles are located in the
"bad halos". Running with premerge=False improves the situation slightly
(slightly fewer "bad halos", better agreement in some of the real ones).

I think this means that the halo finder isn't correctly joining up some
chains across processors, but I'm not familiar enough with the parallelHOP
code to know where to look first. The only changes I've made to the code
are in halo_objects.py, updating the old self.hierarchy.region_strict and
similar calls as well as fixing units in a few places. I haven't made any
changes in parallel_hop_interface.py.

Has anyone run into a similar problem before? If integrating parallelHOP is
known to be unworkable (or extremely difficult) I'll start looking at other
options. Otherwise, if anyone has some insight into where the problem might
lie or what to look at first, I'd appreciate any advice I can get. Thanks.
     - Josh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20160404/8107edbe/attachment.htm>